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Abstract 

The  war  on  terrorism  has  brought  with  it  many  challenges,  one  of  which  being 
combatants  wearing  no  standard  uniform  and  blending  into  the  urban  population. 
To  assist  with  enemy  detection  and  tracking,  imaging  systems  that  acquire  spectral 
information  bring  to  light  many  features  in  a  scene  which  were  once  undetectable. 
Our  research  utilizes  multispectral  technology  to  better  exploit  naturally  occurring 
phenomena  to  identify  combatants. 

In  2008,  the  Sensors  Exploitation  Research  Group  at  the  Air  Force  Institute  of 
Technology  began  using  spectral  properties  of  skin  for  the  detection  and  classihcation 
of  humans.  Since  then,  a  multispectral  skin  detection  system  was  developed  to  exploit 
the  optical  properties  of  human  skin  at  wavelengths  in  the  visible  and  near  infrared 
region  of  the  electromagnetic  spectrum.  A  rules-based  detector,  analyzing  an  image 
spectrally,  currently  bases  its  skin  pixel  selection  criteria  on  a  diffuse  skin  reflectance 
model.  However,  when  observing  skin  in  direct  view  of  the  sun,  a  glint  of  light 
off  skin  is  common  and  indicates  specularity.  The  areas  of  skin  with  a  high  degree 
of  specular  reflectance  result  in  misdetections.  We  show  that  skin  is  characterized 
by  diffuse  and  specular  reflectance,  with  both  components  dependent  on  the  scene 
conhguration.  While  we  cannot  always  rely  on  the  person  to  directly  face  the  camera 
or  have  constant  illumination  conditions,  it  is  important  to  have  flexibility  with  the 
rules-based  detector  as  the  scene  changes.  Our  research  better  characterizes  skin 
reflectance  as  a  function  of  source  and  detector  angular  locations  to  improve  on  the 
rules-based  detector. 

Our  research  approach  first  characterizes  skin’s  specular  reflectance  with  direct 
measurements.  The  fitting  of  a  bidirectional  reflectance  distribution  function  model  to 
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the  measurements  with  approximately  8.2%  error  allows  us  to  incorporate  the  specular 
reflection  component  into  the  existing  diffuse  model.  A  method  for  extracting  surface 
reflectance  of  a  digitized  three  dimensional  subject,  paves  the  way  for  simulating  many 
different  conditions  for  a  representative  detection  scenario.  The  result  is  a  method 
to  model  the  effects  that  changing  scene  configuration  has  on  skin  reflection  and  our 
ability  to  reliably  do  skin  detection. 
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A  MULTISPECTRAL  BIDIRECTIONAL  REFLECTANCE  DISTRIBUTION 
FUNCTION  STUDY  OF  HUMAN  SKIN  FOR  IMPROVED  DISMOUNT 

DETECTION 

1.  Introduction 

Remote  sensing  is  defined  as  “the  field  of  study  associated  with  extracting  in¬ 
formation  about  an  object  without  coming  into  physical  contact  with  it”  [35]. 
Remote  sensing  is  often  used  to  scan  a  scene  for  a  geological  survey  or  gather  intelli¬ 
gence  with  persistent  monitoring  of  a  remote  area  [29,  32],  The  information  gathered 
can  be  used  to  locate  natural  mineral  deposits  or  disrupt  a  planned  terrorist  attack. 

The  human  eye  is  a  sensor  that  is  similar  to  other  optical  sensors  whereby  photons 
are  converted  into  an  electrical  signal  for  information  processing  [14].  While  the  eye 
is  limited  to  visible  wavelengths,  much  more  information  is  available  throughout  the 
electromagnetic  spectrum.  With  the  availability  of  multispectral  and  hyperspectral 
systems,  both  spatial  and  spectral  information  for  a  scene  are  collected  simultaneously. 
With  added  spectral  dimensionality,  objects  that  once  blended  into  a  scene  for  the 
human  eye  now  become  more  distinguishable. 

With  most  passive  sensors  used  remotely,  natural  sources  from  the  environment 
emit  the  observed  radiation.  The  sun  is  often  the  greatest  contributing  source  from 
visible  (VIS)  to  near-infrared  (NIR)  wavelengths.  Factors  affecting  the  radiation 
reaching  a  detector  are  related  to  the  atmospheric  path  and  characteristic  reflectance 
of  in-scene  objects.  With  prior  knowledge  of  an  object’s  reflectance  spectra  and 
atmospheric  compensation  techniques,  an  automated  system  can  £11  the  requirement 
for  the  previously  stated  missions. 
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1.1  Motivation 


The  current  war  on  terror  is  an  irregular  war  against  an  enemy  that  blends  into  the 
population.  The  threat  is  capable  of  concealing  weapons  and  inflicting  harm  on  the 
unsuspecting.  The  INSPIRE  project  is  working  to  identify  radical  extremists  with  the 
integration  of  a  sensor  package  to  analyze  human  motion  [22].  Since  every  individual 
has  a  specihc  walking  pattern  that  changes  when  that  individual  is  carrying  a  load,  the 
INSPIRE  team  is  building  a  database  that  records  the  gait  cycle  of  human  subjects 
with  and  without  a  load.  The  INSPIRE  research  has  the  potential  of  stopping  those 
with  harmful  intent  by  tracking  and  recognizing  a  change  in  gait  cycle. 

The  Civil  Air  Patrol  (CAP)  is  a  volunteer  civilian  auxiliary  to  the  United  States 
Air  Force.  One  of  its  many  missions  is  that  of  search  and  rescue.  An  addition  to 
its  support  equipment  for  performing  the  search  and  rescue  mission  is  the  Airborne 
Real-time  Cueing  Hyperspectral  Enhanced  Reconnaissance  (ARCHER)  imaging  sys¬ 
tem  [6].  The  ARCHER  system  currently  compares  reflected  spectra  to  a  library  of 
previously  measured  data.  Expanding  its  library  to  human  skin  spectra  may  improve 
its  capabilities  for  locating  people. 

Human  detection  is  an  important  hrst  step  to  locating  and  tracking  people’s  move¬ 
ment.  Our  research  is  working  to  save  lives  with  an  improved  method  for  human 
detection. 

1.2  Research  Goals 

To  better  detect  people,  we  must  accurately  detect  skin  [8,  18,  28,  30].  We  accom¬ 
plish  skin  detection  by  exploiting  its  spectral  reflectance  at  specihc  wavelengths  in  the 
VIS  to  NIR  [28].  Limitations  due  to  the  scene  conhguration,  e.g.,  lighting  conditions 
and  human  orientation  to  the  camera,  drove  a  requirement  to  better  characterize  skin 
rehectance  with  changing  conditions. 
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Light  reflecting  off  a  surface  may  exhibit  diffuse  and/or  directional  behavior.  Im¬ 
perfections  on  the  surface  generally  influence  this  behavior  and  is  characterized  by 
a  Bidirectional  Reflectance  Distribution  Function  (BRDF),  that  may  change  with 
wavelength.  As  an  example,  consider  small  pebbles  thrown  toward  a  gravel  surface, 
where  the  pebbles  are  much  smaller  in  size.  The  pebbles  likely  bounce  back  at  ran¬ 
dom  directions,  diffusely,  depending  on  the  slope  of  the  gravel  surfaces  they  impact. 
A  basketball  much  larger  in  size  thrown  at  the  same  surface,  bounces  back  in  a  more 
predictable  direction,  specularly.  While  the  surface  looked  rough  to  the  small  pebbles, 
it  looked  smooth  to  the  basketball.  In  the  same  way  electromagnetic  radiation  strik¬ 
ing  a  surface  reflects  with  wavelength  analogous  to  size  from  the  previous  example 
[15].  In  our  case,  we  are  observing  light  reflecting  from  skin.  Therefore,  characterizing 
surface  reflectance  of  human  skin,  at  our  wavelengths  of  interest,  is  needed  to  better 
quantify  skin  apparent  reflectance. 

Prior  research  in  [28]  modeled  subsurface  skin  reflectance  with  the  limitation  of 
normal  incident  illumination,  resulting  in  a  constant  diffuse  reflectance  irrespective 
of  illumination  conditions.  The  diffuse  reflectance  values  calculated  for  different  skin 
types  are  then  used  as  boundary  conditions  for  a  rules-based  detector.  We  know  from 
Fresnel’s  theory  that  reflectance  at  an  interface  changes  as  incident  angle  moves  fur¬ 
ther  off  normal  to  the  surface  [16].  With  Fresnel,  the  media  refractive  indices  dictate 
the  reflected  light  as  a  function  of  incident  angle.  For  skin,  some  of  the  radiation 
transmits  into  the  subsurface  layers  and  reemerges,  resulting  in  a  combined  subsur¬ 
face  and  surface  reflectance.  The  second  goal  of  our  work  is  to  incorporate  specular 
surface  reflectance  into  the  existing  diffuse  skin  reflectance  model.  Application  of 
the  combined  model  is  performed  on  a  cylinder  surface  representing  the  shape  of  a 
persons  face  and  neck,  and  then  on  a  virtual  scanned  human. 

Our  research  contribution  is  a  more  accurate  skin  reflectance  characterization  for 
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the  changing  scene  conhguration.  Better  characterization  of  human  skin  reflectance 
for  our  VIS  to  NIR  wavelengths  used  for  detection  serves  to  improve  our  spectral 
detection  results  of  distinguishing  between  people  and  other  objects  in  scene. 

1.3  Thesis  Overview 

The  remainder  of  this  thesis  is  organized  as  follows.  Chapter  2  is  a  review  of 
background  material  on  concepts  that  describe  the  nature  of  light,  its  interaction 
with  media,  and  previous  work  on  skin  detection.  The  material  in  Chapter  2  is 
drawn  from  sources  of  literature  by  way  of  the  engineering  and  physics  community 
and  is  referred  to  throughout  the  document.  Chapter  3  has  the  methodology  and 
intermediate  research  results  for  characterizing  human  skin  surface  reflectance.  The 
BRDF  of  human  skin  is  characterized  at  our  detection  wavelengths  and  the  envi¬ 
ronment  for  application  is  described.  Chapter  4  presents  measurements  taken  with 
the  AFIT  SERG  multispectral  system  for  off  normal  illumination.  A  comparison  is 
made  to  a  similar  scenario  simulated  within  animation  rendering  software.  Chapter 
5  summarizes  the  work  accomplished  in  this  thesis,  its  contribution  to  the  research 
subject  of  dismount  detection,  and  recommendations  for  future  work.  Appendices  are 
included  to  show  intermediate  steps  that  are  necessary  to  ensure  accurate  results  in 
the  following  areas:  geometric  equations  converting  the  global  angular  illumination 
onto  a  surface  to  a  localized  value  with  respect  to  a  facet  normal;  BRDF  of  spectralon 
for  different  illumination  conditions  at  633  nm  and  3390  nm;  and  methodology  for 
extracting  scene  reflectance  values  from  Blender  simulations. 
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2.  Background 


Chapter  2  covers  concepts  from  published  literature  that  are  necessary  to  char¬ 
acterizing  skin  reflectance  for  detection.  In  Section  2.1,  radiometric  quantities 
are  introduced  as  a  foundation  to  the  models  developed  for  reflectance.  Section  2.2, 
reviews  optical  concepts  that  apply  to  the  design  of  the  image-based  Bidirectional  Re¬ 
flectance  Distribution  Function  (BRDF)  measurement  system.  In  Section  2.3,  there 
is  an  introduction  to  BRDF  that  includes  a  review  of  relevant  literature  in  modeling 
and  measurement  which  is  later  applied  in  our  research  to  skin.  Section  2.4,  reviews 
the  skin  reflectance  model  in  [28]  developed  by  the  Air  Force  Institute  of  Technol¬ 
ogy  (AFIT)  Sensors  Exploitation  Research  Group  (SERG)  which  is  expanded  on  to 
include  BRDF  in  Ghapter  3.  Finally,  Section  2.5  introduces  image  processing  tools 
used  in  skin  detection. 

2.1  Radiometry 

Radiometry  is  the  study  of  electromagnetic  radiation  measurement.  Radiometric 
quantities  from  [13]  describe  a  flux  density  quantitatively  in  order  to  gather  informa¬ 
tion  about  a  scene.  With  these  quantites,  a  model  is  created  to  represent  the  conhg- 
uration  of  a  remote  sensing  scenario,  and  determine  what  is  observed  by  a  detector 
from  self  emitting  and  reflecting  sources  of  radiation.  Section  2.1  covers  radiometry 
concepts  used  in  our  research  for  measurement  and  modeling  of  skin  reflectance. 

2.1.1  Radiometric  Quantities. 

Within  radiometry,  the  most  fundamental  quantity,  represented  in  energy  units, 
is  power  or  flux  (<he)  in  watts  (W).  The  convention  for  choosing  units  is  to  relate 
them  to  the  type  of  detector  used  for  measurement,  e.g.,  photon  detectors  respond 
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directly  to  electron  generation  from  incident  photons.  Quantities  measured  with 
photon  detectors  are  set  to  photon  units  (<hp)  in  photons/s.  With  the  following 
relationship,  energy  units  are  converted  to  photon  units: 


(2,1,1) 


where  c  is  the  constant  for  the  speed  of  light,  h  is  Plank’s  constant,  and  A  is  wave¬ 
length. 

The  radiometric  quantities  that  facilitate  in  measurement  and  modeling  are  exi- 
tance,  irradiance,  and  radiance.  Exitance  (Mg)  in  W/cm^  is  the  flux  leaving  a  source 
area  (A*),  dehned  as: 

Me  =  (2.1.2) 

Irradiance  {Ee)  in  W/cm^  is  the  flux  falling  on  a  surface  area  (A^),  dehned  as: 

E,  =  (2.1.3) 

Ad 


The  hux  leaving  a  source  area  per  solid  angle  (fid),  measured  in  steradians  (sr),  is 
radiance  (Le)  in  W/cm^-sr  with  a  cosine  correction  of  9s  for  off  normal  solid  angles, 
dehned  as: 


Le 


As  cos  9s  ■  fid 


(2.1.4) 


In  Equation  (2.1.4),  a  small  angle  approximation  is  assumed  for  A5<^R^,  where  R 
is  the  range  from  the  source  area  to  the  detector.  Without  the  small  angle  approx¬ 
imation,  hux  is  changing  with  respect  to  surface  area  and  solid  angle  represented 
by: 


cos9sdAsdQ.d ' 


(2.1.5) 


Given  a  known  radiance,  any  other  radiometric  quantity  is  found  by  modifying  Equa- 
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tion  (2.1.5).  As  seen  in  Equation  (2.1.6),  flux  is  isolated  on  the  left  side  of  the  equal 
sign  as: 


COS  OgdAgd^ld, 


Qd 


(2,1.6) 


thereby  allowing  the  right  side  to  be  placed  into  Equation  (2.1.2)  or  (2.1.3).  With 
the  flux  substitution  and  a  known  scenario  conhguration,  exitance  and  irradiance  are 
solved  from  the  known  radiance. 

While  radiometric  quantities  vary  with  location,  they  also  vary  with  wavelength. 
Referred  to  as  spectral  radiance  (Le(A)),  the  total  quantity  is  calculated  at  a  specihc 
wavelength  or  integrated  over  the  wavelength  region  of  interest.  The  total  radiance 
over  the  wavelength  region  of  interest  is  calculated  as: 


Le 


j  Le(A)dA. 

A2 


(2,1.7) 


Demonstrated  in  Section  2.1  is  the  flexibility  of  the  fundamental  radiometric  terms. 
When  transforming  the  quantities  as  shown,  a  model  is  created  based  on  what  is 
known  and  what  is  assumed. 


2.1.2  Self  Emitting  Sources. 

According  to  blackbody  theory,  an  object  at  a  specihc  temperature  emits  radi¬ 
ation  as  function  of  wavelength.  The  blackbody  spectral  radiance  for  an  object  at 
temperature  (Tobj)  is  characterized  by  the  following  equation: 


LeW 


2hc^ 


A5 


he 


(2.1.8) 
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(a)  (b) 


Figure  1.  Blackbody  spectral  radiance  plots  in  energy  units,  for  temperatures  repre¬ 
senting  (a)  the  sun  at  5950  K  and  (b)  skin  at  305  K. 


The  actual  radiation  emitted  from  an  object  is  calculated  as  a  fraction  of  the  black- 
body  with  the  parameter  emissivity  (e).  The  emissivity  is  either  constant  or  varies 
with  wavelength.  In  Figure  1,  blackbody  spectral  radiance  is  plotted  for  5950  K  and 
305K,  temperatures  for  the  sun  and  human  skin.  It  is  apparent  that  lower  tempera¬ 
tures  peak  at  higher  wavelengths  and  higher  temperatures  peak  at  lower  wavelengths. 
Self  emitted  radiation  belonging  to  skin  is  negligible  in  the  VIS  to  NIR  wavelength 
region,  while  the  self  emitted  radiation  belonging  to  the  sun  contributes  mostly  in 
the  VIS  to  NIR  region. 


2.1.3  Media  Apparent  Properties. 

When  electromagnetic  radiation  is  incident  on  a  media,  it  is  partly  reflected, 
absorbed,  and  transmitted.  The  percent  of  incident  flux  transformed  is  represented 
with  the  terms  reflectance  (p),  absorptance  (a),  and  transmittance  (r)  dehned  as: 


P 


^reflected 

^incident 


(2.1.9) 
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^absorbed  ; 

a  =  — - ,  ana 

^incident 
^transmitted 


T  = 


$ 


incident 


Conservation  of  energy  dictates  that: 


(2.1.10) 

(2.1.11) 


1  =  p  +  a  +  r, 


(2.1.12) 


and  according  to  Kirchoff’s  law,  in  order  for  the  object  remain  in  thermal  equilibrinm, 
its  emittance  eqnals  absorbtance  [13]. 

The  terms  in  Eqnation  (2.1.12)  represented  spectrally,  are  instrnmental  to  deter¬ 
mining  the  flnx  transfer  from  a  sonrce  to  a  detector.  Ultimately,  the  observed  flnx 
density  is  characteristic  of  these  media  properties  with  which  it  interacts. 


2.2  Optics 

Geometric  optics  nses  a  treatment  of  radiation  that  is  ideally  imaged  when  propa¬ 
gated  throngh  a  system,  where  a  point  in  the  object  plane  maps  perfectly  to  a  point  in 
the  image  plane.  Section  2.2  describes  the  optical  components  and  theory  to  describe 
the  propagation  of  light  for  imaging  system  design  and  radiative  transfer. 


2.2.1  Optical  Components. 


A  lens  and  mirror  are  optical  components  that  converge  or  diverge  collected  light. 
These  components  are  nsed  to  image  an  object  to  a  location  described  by  the  thin 
lens  eqnation  in  [16]: 


1 


+ 


1 

So 


1 

7’ 


(2.2.1) 
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where  So  is  the  object  distance,  s*  is  the  image  distance,  and  /  is  the  focal  length. 
The  magnihcation  (M)  for  the  setup  of  a  single  optic  is: 


M  =  -  —  .  (2.2.2) 

An  object  area  Aobject  is  imaged  to  an  area  Aimage  through  an  optical  system  with  a 
magnihcation  as: 

Aimage  =  '  Aobject-  (2.2.3) 

With  these  equations,  optical  components  are  positioned  to  create  an  imaging  system. 
A  focal  plane  array  (FPA)  placed  at  the  image  plane  captures  information  from  the 
object  plane  for  the  instantaneous  held  of  view  of  each  individual  pixel. 


2.2.2  Fresnel  Equation. 

With  Fresnel  equations,  the  amount  of  light  rehecting  from  the  front  surface  at 
the  interface  between  two  dielectric  media  is  calculated  [16].  The  two  media  have 
indices  of  refraction  ni  and  n2,  which  may  change  with  wavelength.  The  rehectance 
of  incoming  light  normal  to  a  dielectric  surface  is  calculated  as: 


Po 


/  na  -  niy 
\n2  +  nij 


(2.2.4) 


At  incident  angles  other  than  normal  to  the  surface,  the  rehectance  calculation  for  S 
and  P  polarization  components  are: 


Ps{0i) 


Til  COS  6i  —  n2  cos  6t 
ni  cos  9i  +  na  cos  9t 


2 

and 


(2.2.5) 


Pp{9i) 


\ni  cos9t  +  na  ' 


(2.2.6) 
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Figure  2.  Reflectance  calculated  with  Fresnel  equations  for  the  air  to  skin  interface, 
showing  S  polarized  (blue),  P  polarized  (green),  and  unpolarized  light  (red)  dependent 
on  incident  angle. 

For  unpolarized  light,  the  reflectance  is  the  average  of  Equations  (2.2.5)  and  (2.2.6). 
In  [28],  an  air  (ni  =  1.0)  to  skin  (n2  =  1.5)  interface  is  calculated  with  Equation  (2.2.4), 
resulting  in  a  reflectance  of  0.04.  The  off  normal  reflectance  for  the  same  air  to  skin 
indices  applying  Equations  (2.2.5)  and  (2.2.6)  is  a  function  of  incident  angle,  and  is 
plotted  in  Figure  2. 

Unpolarized  reflectance  calculated  with  Fresnel  equations  is  often  approximated 
with  the  following  Schlick’s  model  [34]: 

PSchUckipo,  Oi)  =  po  +  {l-  Po)(l  -  cos(6'i))®  (2.2.7) 

where  po  is  reflectance  defined  in  Equation  (2.2.4)  and  9i  is  the  illuminated  inci¬ 
dent  angle.  In  Figure  3,  the  Schlick  approximation  is  shown  in  comparison  to  the 
unpolarized  reflectance  calculation  from  Fresnel  dielectric  equations. 
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Figure  3.  Unpolarized  Fresnel  reflectance  (green)  for  the  air  to  skin  interface,  compared 
with  Schlicks  approximation  (blue)  based  on  a  normal  incident  reflectance  of  0.04. 


2.3  Bidirectional  Reflectance  Distribution  Function  (BRDF) 

Most  surfaces,  natural  and  man  made,  are  not  perfectly  smooth.  At  a  microscopic 
scale,  imperfections  are  seen  along  material  surfaces.  These  surfaces  reflect  light 
to  a  location  that  is  dependent  on  the  angular  direction  of  incoming  light  and  the 
imperfection  orientation.  Meanwhile,  a  detector  collects  reflected  light  as  a  function  of 
its  size  and  location.  An  accurate  characterization  of  material  reflectance  with  these 
considerations  is  necessary  for  best  estimation  of  apparent  reflectance.  The  BRDF 
gives  that  characterization  as  a  function  of  light  source  and  detector  conflguration. 

2.3.1  BRDF  Background. 

Three  types  of  reflection  are  specular,  diffuse,  or  a  combination  of  specular  and 
diffuse  components.  Specular  reflection  occurs  with  directional  reflection  of  light 
from  the  surface,  with  incident  angle  equal  to  reflection  angle.  A  material  which 
best  reflects  specularly  is  a  mirror.  Diffuse  reflection  occurs  with  an  equal  spread 
of  reflecting  light  in  all  directions  from  a  surface.  A  theatre  projector  screen  is  an 
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Figure  4.  Diagram  of  of  incoming  irradiance  and  outgoing  radiance  in  spherical  coordi¬ 
nates  from  [5].  The  out  of  plane  nature  with  (j)i  and  about  the  x  axis  is  shown  along 
with  the  indicated  incident  9i  and  reflected  0^  angles  from  the  surface  normal  n. 


example  of  a  diffuse  material  designed  for  all  observers  in  the  theatre  to  see  the  same 
image.  The  diffuse  and  specular  combination  has  a  spread  of  reflecting  light  about 
an  illuminated  surface,  with  an  additional  specular  component.  Most  other  surfaces 
from  metal  to  vegetation  fall  into  the  diffuse  and  specular  combination  category. 

BRDF  {Jerdf)  in  [36],  characterizes  reflectance  with  the  following  dehnition: 


fERDpi^i)  (t>ii  A) 


A 


Lr  )  '^) 


(2.3.1) 


where  the  radiance  leaving  a  surface  over  the  irradiance  onto  the  surface  results  in 
a  reflectance  per  steradian  measure,  with  associated  angles  shown  in  Figure  4.  The 
BRDF  is  a  distribution  over  the  hemisphere  and  integrated  over  the  detector  solid 
angle  to  calculate  the  apparent  reflectance.  A  common  measure  of  a  diffuse  material  is 
the  reflectance  over  the  entire  hemisphere,  named  Directional  Hemispheric  Reflectance 
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(DHR); 


„  TT 

2'k  2 

PDH Ri^ii  J  J  f COS  9r  siliOrdOrdcpr-  (2.3.2) 

0  0 

2.3.2  BRDF  Measurement  Techniques. 

Measuring  the  complete  BRDF  of  a  material  is  time  consuming.  In  most  cases, 
conditions  are  chosen  to  suit  the  application.  BRDF  is  a  subset  of  the  bidirectional 
scatter  distribution  function  which  includes  reflectance  and  transmittance.  For  our 
application,  we  do  not  measure  transmittance  since  we  only  need  to  characterize 
reflectance.  Our  simplihcation  eliminates  an  entire  hemisphere  of  measurements  on 
the  other  side  of  a  surface.  The  BRDF  can  also  vary  over  a  spectrum  of  wavelengths. 
Specihc  wavelengths  of  interest  are  normally  chosen  to  limit  the  amount  of  time 
needed  to  measure  all  wavelengths  over  an  area  of  the  spectrum  of  interest.  Even 
with  the  described  subset  of  conditions,  it  can  take  several  days  to  collect  data.  For 
our  research,  it  is  not  practical  for  a  living  subject  to  sit  still  for  an  extended  period 
of  time  while  data  is  collected.  Section  2.3.2  reviews  published  literature  describing 
some  of  the  current  BRDF  measurement  techniques. 

A  gonioreflectometer  is  a  specialized  measurement  device  used  to  collect  BRDF 
data.  The  major  components  of  the  gonioreflectometer  setup  are  the  radiating  source 
with  optics,  motorized  armature  with  detector,  mounted  sample,  and  data  acquisition 
equipment.  With  the  system,  the  detector  rotates  around  the  sample  with  the  source 
set  to  a  hxed  incident  angle.  When  collecting  data,  the  reflected  energy  is  collected 
over  the  90  degrees  from  the  sample  normal.  The  process  is  repeated  for  each  angle 
of  incidence  over  the  90  degrees  about  the  sample.  For  out  of  plane  measurements,  a 
full  180  degree  span  is  covered  for  each  set  of  incident  and  reflected  angles.  With  10 
degree  angular  steps,  it  adds  up  to  1,458  data  points  for  each  wavelength  of  interest. 
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Figure  5.  Image  from  [37]  demonstrating  the  application  of  an  image-based  measure¬ 
ment  system  for  characterizing  BRDF  of  a  cylindrical  shape.  The  camera  rotates 
around  the  cylinder,  collecting  reflecting  light  from  a  stationary  source. 

A  higher  fidelity  data  collection  at  1  degree  steps  increases  to  1,458,000  data  points 
at  each  wavelength  of  interest. 

A  novel  approach  from  [12,  37]  uses  an  image-based  technique  to  capture  many  of 
these  data  points  simultaneously,  which  reduces  the  data  collection  time.  By  intro¬ 
ducing  a  camera  with  an  array  of  pixels  to  replace  a  single  detector,  each  pixel  acts 
as  a  detector  for  the  BRDF  measurement  seen  in  Figure  5.  In  [37],  they  illuminate 
isotropic  curved  samples  and  use  a  camera  to  take  pictures  as  they  rotate  the  camera 
around  the  sample.  With  the  symmetry  of  the  sample  and  the  camera  pixel  specifi¬ 
cations  known,  they  map  the  individual  pixels  to  an  area  on  the  sample.  They  then 
calibrate  the  source  and  detector  to  determine  the  reflectance  per  steradian.  They 
claim  any  curved  surface  can  be  sampled  with  the  use  of  a  range  scanner  to  exactly 
describe  the  object  orientation. 

In  [12] ,  they  use  a  parabolic  mirror  to  direct  incident  rays  to  a  single  point  on  the 
sample  located  at  the  focal  point  of  the  mirror  seen  in  Figure  6.  Scattered  rays  from 
the  sample  collimate  upon  contact  with  the  parabolic  mirror.  A  camera  is  positioned 
in  the  path  of  these  collimated  rays  to  collect  the  data.  The  camera  pixels  are  mapped 
to  the  parabolic  mirror  to  determine  reflection  angle  from  the  sample.  The  incident 
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TOP  VIEW 


Figure  6.  Image  from  [12]  demonstrating  the  application  of  an  image-based  measure¬ 
ment  system  for  characterizing  BRDF  onto  a  flat  sample  placed  at  the  focal  point  of 
a  parabolic  mirror.  The  rays  reflecting  from  the  sample  at  the  focal  point  collimate 
when  incident  on  the  mirror,  which  is  then  directed  to  the  camera. 

ray  can  be  shifted  to  change  the  incident  angle  directed  to  the  sample. 

In  order  to  accurately  estimate  radiometric  quantities  for  in-scene  objects,  tech¬ 
niques  described  in  Section  2.3.2  are  applied  in  [37,  40,  41]  for  measuring  skin  BRDF. 
Since  it  is  nearly  impossible  to  measure  every  condition,  models  are  £t  to  the  lim¬ 
ited  amount  of  measured  data  to  describe  the  response.  The  next  section  focuses  on 
common  models  representing  BRDFs. 

2.3.3  BRDF  Modeling. 

The  need  for  high  fidelity  BRDF  models  has  been  largely  driven  by  the  computer 
graphics  community,  which  is  seen  in  recent  animated  movies.  The  goal  of  the  movie 
industry  is  to  render  realistic  three  dimensional  images  from  in-scene  objects  with  pre¬ 
defined  surface  characteristics.  Within  our  research,  we  work  backwards  by  extracting 
the  apparent  properties  of  objects  from  a  measured  or  rendered  image  [26].  In  [17,  23, 
25] ,  skin  reflectance  is  modeled  from  subsurface  light  transport  equations  using  Monte 
Carlo  methods.  Our  method  for  modeling  skin  utilizes  a  BRDF  for  characterizing  the 
skin  surface  reflectance.  First,  some  of  the  techniques  commonly  used  for  BRDF 
modeling  are  presented. 

The  simplest  BRDF  model  to  approximate  surface  reflectance  is  the  Lambertian 
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model  [24]: 


Id  =  -■  (2.3.3) 

71 

The  Lambertian  model  assumes  diffuse  light  is  scattered  equally  in  all  directions  and 
not  a  function  of  position. 

The  microfacet  model  in  [31]  describes  a  rough  surface  as  made  up  of  microfacets 
with  surface  normals  distributed  with  a  Gaussian  distribution  and  utilized  in  the 
following  form; 


fs{9i,9r,A(f),  A) 


PSchlick{^Po{^I) )  9rj  ^0)) 

87r(j2(A)cos^6*(6'i,  9r,  A0)  cos  9i  cos  9r 


— tan^  8(6 

2cr2(A) 


(2.3.4) 


with  0  and  9  defined  as: 


/3(9i,9r,A(l))  = 


cos  ^(cos(6*i)  cos(6'r)  +  sin(0j)  sin(6'r)  cos(A0)) 


(2.3.5) 


9{9i,  9r,  A0)  =  cos 


cos(6*i)  +  cos(6'r) 


(2.3.6) 


'2cos{/3{9i,9r,  A(j))) 

The  angles  denoted  9i  and  9r  are  the  global  incident  and  reflected  angles  to  the 
surface,  (3(9i,  9r,  A0)  is  the  angle  of  incidence  onto  a  microfacet,  and  tan(6'(6*j,  9r,  A0)) 
is  the  local  surface  slope.  The  spectral  terms  for  slope  variance  ct^(A)  represents 
roughness  and  Po(A)  normal  incident  reflectance  are  fit  for  a  specific  material.  Each 
microfacet  obeys  Fresnel  equations  approximated  by  Schlick’s  model  for  unpolarized 
light  represented  by  ps,  and  Snell’s  Law  of  reflection  for  light  about  each  local  facet 
normal.  Figure  7  demonstrates  the  in-plane  plots  of  the  microfacet  model  for  0,  20, 
40,  and  60  degree  incident  angles  and  variance  of  0.01,  0.02,  and  0.03.  In  the  Figure  7 
plots,  a  constant  normal  reflectance  of  0.04  is  chosen  for  the  Gaussian  function,  but 
for  a  more  accurate  representation  the  parameters  must  be  fit  to  measured  data.  In 
[39],  Ward  implemented  the  Gaussian  model  as  a  simple  and  physically  representative 
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(a) 


(b) 


(c) 


Figure  7.  In-plane  microfacet  model,  with  set  normal  reflectance  of  0.04  and  variance 
of  (a)  0.01,  (b)  0.02,  and  (c)  0.03.  The  four  different  incident  angles  represented  on 
each  plot  are  0  deg  (dark  blue),  20  deg  (green),  40  deg  (orange),  and  60  deg  (light 
blue). 


rendering  tool  in  graphics  software. 

Additional  models  relying  on  physical  principals  and  data  htting  are  found  in 
[10,  11,  21,  33,  38].  The  models  in  [10,  11,  21,  33,  38]  add  additional  hdelity  to  the 
microfacet  model.  The  Torrence- Sparrow  model  adds  a  factor  that  accounts  for  shad¬ 
owing  from  neighboring  facets.  The  Maxwell-Beard  model  introduces  a  volumetric 
component  that  is  attributed  to  subsurface  reflectance.  The  Cook- Torrence  model 
adds  an  ambient  term  from  indirect  sources.  Over  time,  these  models  have  been  in¬ 
tegrated  into  software  packages  such  as  Blender®  and  DIRSIG  [2,  3].  With  Blender 
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and  DIRSIG,  an  object  is  given  reflectance  characteristics  for  scenario  modeling. 


2.4  Human  Skin 

Skin  is  a  complex  material  made  np  of  different  layers  and  physical  parameters  that 
affect  reflectance,  shown  in  Fignre  8.  The  general  strnctnre  of  skin  is  the  epidermis, 
dermis,  and  snbcntaneous  tissue.  The  epidermis  is  subdivided  into  five  layers  named 
the  stratum  corneum,  stratum  lucidum,  stratum  granulosum,  stratum  spinosum,  and 
stratum  basale.  The  dermis  is  subdivided  into  four  layers  named  the  papillary  der¬ 
mis,  upper  blood  net  dermis,  reticular  dermis,  and  deep  blood  net  dermis.  Within  the 
layers  are  fractions  of  physical  parameters  of  blood,  water,  collagen,  melanosomes, 
hemoglobin,  bilirubin,  and  betacarotene.  Each  layer  described  with  these  underlying 
components  has  a  response  to  radiation  that  sums  together  to  generate  overall  re¬ 
flectance.  Section  2.4  reviews  the  literature  from  [28]  to  describe  the  individual  layer 
properties  and  build  the  existing  skin  reflectance  model. 

2.4.1  Optical  Properties. 

The  radiation  transfer  through  each  layer  is  affected  by  absorption  and  scattering. 
Scattering  occurs  when  traveling  radiation  impacts  with  a  particle  that  is  significant 
enough  in  size  to  cause  a  deviation  in  its  course  [16].  Scattering  is  dependent  on 
wavelength  of  the  traveling  wave  and  the  size  of  the  particle  it  impacts.  Different 
types  of  scattering  contribute  at  different  wavelengths  such  as  Rayleigh  and  Mie 
scattering.  To  model  scattering  for  each  layer,  a  power  law  is  fit  in  [28]  allowing 
flexibility  with  a  and  b  fitting  parameters  for  each  human  subject: 

s(A)  =  a■A-^  (2.4.1) 

The  total  absorption  is  additive  of  the  absorption  characteristics  of  physical  pa- 
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Figure  8.  Layers  of  skin  used  in  the  Kubelka-Munk  model  for  calculating  diffuse  re¬ 
flectance  [28]. 


rameters  in  each  layer.  The  fraction  of  these  parameters  is  unique  for  each  individual 
person.  The  concentration  parameters  as  a  percentage  of  the  n*^  layer  of  skin  are  rep¬ 
resented  by  blood  (b„),  water  (w„),  collagen  (c„),  melanosomes  (v„),  and  oxygenation 
fraction  of  blood  (7).  Spectral  absorption  tables  with  units  of  cm“^,  empirically  de¬ 
rived  in  [28]  are  collagen  (aco«(A))  ,  water  (a^at(^)),  betacarotene  (acar(A)),  bilirubin 
(ai,ii(A)),  oxygenated  hemoglobin  {aohbi{A)),  and  deoxygenated  hemoglobin  {adhbi{A)). 
The  absorption  for  melanosomes  (amei(A)),  derived  in  [28]  is: 


amei  (A)  =  6.6  ■  10^^ (2.4.2) 

The  stratum  corneum  layer  total  spectral  absoprtion  in  cm“^  is  a  combination  of 
spectral  absorption  from  percent  concentrations  of  water,  collagen,  and  betacarotene 
is  described  as: 

^i(A)  ci(Zco/(A)  T  tci(z^^i(A)  T  (ice(A)  (2.4.3) 
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For  the  epidermis,  ace(A)  is  the  typical  betacarotine  spectral  absorption  in  those  layers 
with  units  of  cm“^.  Layers  2  to  5  consist  of  percent  concentrations  with  spectral 
absorption  from  water,  collagen,  and  betacarotene  described  as: 

®n(A)  ^n®mez(A)  “1“  C^ncoz(A)  “1“  “1“  ®ce(A)  (2.4.4) 

Melanosomes  are  only  in  layers  4  and  5,  all  other  layers  equals  zero.  The  dermal 
layers  of  6  to  9  consist  of  percent  concentrations  with  spectral  absorption  in  cm“^ 
of  water,  collagen,  oxygenated  hemoglobin,  deoxygenated  hemoglobin,  bilirubin,  and 
betacarotene,  and  they  are  described  as: 

®n(A)  CfjOcoz(A)  T  (iCn  4“ 

+  (7®o/iZ)(A)  +  (1  —  '^)cidhb{^)  +  O-carW  +  (2.4.5) 

Equations  (2.4.3),  (2.4.4),  and  (2.4.5)  describe  all  the  absorption  features  needed 
for  subsurface  skin  reflectance  modeling.  Equation  (2.4.1)  is  used  for  scattering  in  all 
layers  except  for  the  reticular  dermis  which  is  scaled  by  the  fraction  of  collagen  in 
that  layer. 

The  absorption  contributes  to  the  characteristic  shape  of  the  overall  reflectance 
and  is  an  indicator  when  detecting  a  medium.  At  higher  wavelenghts  near  1400  nm, 
water  is  highly  absorptive  and  the  reflectance  of  skin  is  low.  Another  key  feature  is  a 
characteristic  W  shape  near  560  nm,  due  to  oxygenated  hemoglobin  seen  in  Figure  9. 
The  cadaver  samples  presented  in  [28]  do  not  have  a  W  feature  due  to  the  lack  of 
oxygenated  blood.  In  Figure  9,  the  shade  of  a  person’s  skin  is  also  evident  and  is 
directly  related  to  melanosome  content.  Type  V/VI  skin,  dehned  in  Table  1,  has  a 
higher  melanosome  level  leading  to  higher  absorption  in  the  visible  wavelengths,  while 
a  lighter  toned  person  is  just  the  opposite. 
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Figure  9.  Radiometer  measured  skin  reflectance  of  normally  incident  light,  for  Type 
I/II  (blue)  and  Type  V/VI  (green)  skin. 


Table  1.  Skin  appearance  as  defined  by  the  Fitzpatrick  scale  [27]. 


Skin  Type 

Skin  Color 

/ 

Very  Fair 

II 

Fair 

III 

White  to  Olive 

IV 

Brown 

V 

Dark  Brown 

VI 

Black 
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These  physical  parameters  play  a  key  role  in  the  development  of  the  diffuse  model 
component.  All  the  spectral  absorption  data  is  documented  in  [28]  and  is  needed  to 
determine  subsurface  skin  reflectance.  The  next  section  describes  how  the  parameters 
are  incorporated  into  the  existing  diffuse  reflection  model. 


2.4.2  Diffuse  Reflectance  Modeling. 

The  Kubelka-Munk  model  from  [9]  describes  the  propagation  of  radiation  through 
an  isotropic  slab  of  thickness  d.  The  theory  is  applied  to  each  of  the  nine  layers  of 
skin,  and  added  with  the  surface  normal  Fresnel  reflectance  to  generate  a  total  re¬ 
flectance  from  the  skin.  The  absorption  and  scattering  transport  coefficients  of  Equa¬ 
tions  (2.4.1),  (2.4.3),  (2.4.4),  and  (2.4.5)  are  related  to  Kubelka-Munk  parameters 
with  the  following  relationship; 


An(A)  = 


®n(A) 


1(1 


^n(A) 


Sn(A)+an(A) 


(2.4.6) 


Sn{\)  = 


'5n(A) 


4  +  38  (  1 
3  ^  45  ' 


(A) 


(2.4.7) 


•5n(A)+an(A)  ^ 

Equations  (2.4.6)  and  (2.4.7)  are  the  Kubelka-Munk  transformed  absorption  and  scat¬ 
tering  coefficients  that  correspond  to  the  specihc  case  of  an  isotropic  media.  The 
Kubelka-Munk  equations  for  reflectance  and  transmittance  of  a  single  layer  are: 


?^n(A) 

T‘n(A) 


_ Smh{Sn{\)ynWdn) _ 

a;n(A)  cosh(S'„(A)|/„(A)d„)  -h  |/n(A)  sinh(S'„(A)|/„(A)d„)  ’ 

Xn{\)  cosh{S n{\)ynWdn)  +  l/n(A)  sinh(S'„(A)|/„(A)d„)  ’ 


(2.4.8) 


(2.4.9) 
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Figure  10.  Conceptual  drawing  of  suface  and  subsurface  reflectance  from  multi-layered 
skin.  The  Kubelka-Munk  model  calculates  a  diffuse  reflectance  value  that  accounts  for 
characteristics  of  each  layer  with  the  resulting  energy  emerging  diffuse  as  it  would  from 
a  bulk  material  [28].  The  BRDF  is  used  to  calculate  the  specular  reflectance  component 
dependent  on  the  surface  geometry. 


where  x  and  y  are  defined  as: 


A„(A)  +  S'„(A) 

^n(A) 


(2.4.10) 


yn{\)  =  -  1. 


(2.4.11) 


While  the  refiectance  and  transmittance  of  a  layer  is  calculated  from  Equations  (2.4.8) 
and  (2.4.9),  there  are  additional  refiectance  and  transmittance  terms  from  the  infinite 
interface  bounces.  When  accounting  for  the  additional  bonces,  the  resulting  recursive 
equations  for  the  accumulated  refiectance  and  transmittance  of  a  single  layer,  seen  in 
Figure  10,  are: 


-Rn+l(A) 


T^(A)V„+i(A) 

1  -  r„+i(A)i?„(A)’ 


r.+i(A) 


T„(A)r„+i(A) 

1  -  r„+i(A)i?„(A)‘ 


(2.4.12) 

(2.4.13) 
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The  term  for  reflectance  in  Equation  (2.4.12)  is  based  on  the  transmittance  and  re¬ 
flectance  from  the  previous  layer.  The  initial  reflectance  (i?o)  is  at  the  skin  surface 
and  is  calculated  as  normal  illuminated  reflectance  po  from  Fresnel  Equation  (2.2.4). 
The  transmittance  (Tq)  into  the  hrst  layer  is  calculated  from  1  -  Rq.  The  calcula¬ 
tions  continue  with  the  next  layers  until  we  reach  the  subcutaneous  fat  which  has 
reflectance  from  measured  values  in  [28] .  The  result  is  a  reflectance  that  emerges  and 
is  characteristic  of  all  skin  layers. 

2.4.3  Fitting  Model  to  Measurements. 

The  Kubelka-Munck  model  developed  in  [28],  has  physical  parameters  that  are 
£t  to  a  specific  person.  Adjusting  these  parameters  allows  us  to  generate  reflectance 
values  that  match  with  measured  data  creating  a  general  model  for  that  person.  The 
fitting  parameters  are  melanosomes  fraction,  blood  level,  blood  oxygenation,  reticular 
dermis  depth,  collagen  fraction,  and  subcutaneous  scale.  The  physical  parameters 
kept  constant  are  the  layer  depths  (excluding  reticular  dermis),  water  percentage  in 
each  layer,  and  blood  percentage.  In  [28],  these  parameters  were  shown  to  be  standard 
values  for  each  person. 

To  fit  a  model  to  measured  data,  a  quantitative  method  is  used  to  provide  a 
measure  of  difference  between  the  them.  A  common  error  function  is  the  root  mean 
square  error  (RMSE): 

RMSE(\)  =  (.P(.^')model  measure)  (2.4.14) 

which  is  applied  repeatedly,  while  changing  parameter  values  until  the  minimum 
error  between  the  measured  and  modeled  reflectance  is  found.  This  approach  use  and 
optimization  search  strategy  with  matlab.  In  Equation  (2.4.14),  N  represents  the 
number  of  wavelength  bands  the  reflectance  is  summed  over. 
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2.5  Radiative  Transfer  and  Detection 


Since  all  media  have  their  own  unique  spectral  properties,  the  process  of  detecting 
a  specihc  medium  is  made  easier  by  collecting  the  spectral  content  of  a  scene.  A  prob¬ 
lem  with  searching  a  scene  in  order  to  locate  a  specific  signature  is  that  illumination 
conditions  and  path  losses  affect  what  is  observed  by  the  detecting  system.  Section 
2.5  reviews  image  processing  techniques  and  radiative  transfer  modeling  supporting 
skin  detection. 

2.5.1  Radiative  Transfer  Modeling. 

Radiative  transfer  modeling  is  a  method  for  predicting  remotely  observed  spectral 
radiance  based  on  scene  and  atmospheric  properties.  Characterizing  a  remote  sensing 
scenario  with  radiometric  terms  allows  tracking  of  ffux  along  a  path  from  the  source 
to  the  detector. 

The  operating  band  of  the  detector  is  important  for  modeling  sources  of  radiation 
affecting  the  system.  Looking  back  at  the  blackbody  radiation  curve  in  Figure  1, 
it  is  clear  the  sun  dominates  in  the  visible  and  NIR  bands  while  lower  temperature 
self  emitting  objects  contribute  most  from  the  NIR  to  long  wave  infrared.  Addi¬ 
tionally,  path  losses  are  attributed  to  the  atmospheric  conditions  which  change  daily. 
MODTRAN  is  a  modeling  tool  that  can  generate  path  radiance  and  losses  for  the 
specified  atmospheric  conditions.  With  these  considerations,  a  simple  diffuse  model 
for  radiance  at  the  aperature  of  a  visible  to  NIR  detection  system,  estimating  medium 
self-emission  negligible,  is  defined  as: 

Le{X)  =  Ta{\)  ■  ^  ■  E,{X)  +  La{X),  (2.5.1) 

71 

where  La  and  Ta  are  atmospheric  radiance  and  transmittance,  Ei  is  incident  irradiance 
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onto  the  medium  surface,  and  p  is  the  medium  reflectance.  In  [30],  the  radiometric 
model  used  for  our  multispectral  skin  detection  system  is: 


1700nm 


^p— pixel 


L  e.—target{^^^^pixel-^op  filter  (2.5.2) 


400nm 


where  Tatm  is  atmospheric  transmittance,  Topt-futer  is  optical  hlter  transmittance.  Tint 
is  integration  time,  ^pixei  is  pixel  solid  angle,  Aopt  is  area  of  the  optic,  and  p  is  quantum 
efficiency.  For  the  specihc  case  of  a  Lambertian  target,  the  radiance  is  substituted 
with: 


Le—targetii^')  —targeti^^) 


pW 


TT 


(2.5.3) 


2.5.2  Empirical  Line  Method  (ELM). 

While  path  characteristics  of  a  scene  are  not  always  known,  the  Empirical  Line 
Method  (ELM)  was  developed  to  estimate  and  compensate  for  these  path  effects.  In 
Equation  (2.5.1),  a  general  radiative  transfer  model  was  introduced  with  variables 
that  are  lumped  into  the  following  form: 


Le(A)  =  a(A)-p(A)  +  5(A). 


(2.5.4) 


For  two  in-scene  media  of  known  reflectance  and  measured  radiances,  the  remaining 
linear  constants  a  and  b  are  solved  as: 


a(A) 


L2(A)  —  Li(A) 
d2(A)  -pi(A)  ’ 


(2.5.5) 


b{\) 


-hi(A)  ■  P2(A)  —  L2(A)  ■  pi(A) 
P2(A)  -pi(A) 


(2.5.6) 
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For  uniform  illumination  of  the  scene,  the  remaining  measured  data  is  mapped  from 
radiance  to  reflectance  as: 


pW 


Le{\)  -  b{\) 

a(A) 


(2.5.7) 


2.5.3  Skin  Detection. 

2.5.3. 1  Basic  Skin  Detector. 


While  observing  landcover  with  space-based  imaging  systems,  a  efficient  method 
was  developed  that  indicated  vegetation  named  Normalized  Difference  Vegetation 
Index  (NDVI)  [4],  As  seen  in  Figure  9,  there  are  characteristic  dips  specihc  to  skin, 
similar  to  vegetation,  which  lead  to  the  development  of  the  Normalized  Difference  Skin 
Index  (NDSI)  in  [28].  Identifying  the  large  difference  in  skin  reflectance  at  lOSOnm 
and  1580nm  drove  the  following  form  of  NDSI; 


p(1080nm)  —  p(1580nm) 
p{1080nm)  +  p(1580nm)  ’ 


(2.5.8) 


where  large  NDSI  values  are  an  indicator  of  skin.  As  a  secondary  measure  for  detecting 
skin,  the  Normalized  Difference  Green  Red  Index  (NDGRI)  is  used.  Since  there  is  a 
large  difference  in  reflectance  between  the  red  and  green  colors  of  skin  it  is  defined 
as: 


/S 


p(633nm)  —  p(544nm) 
p(633nm)  -|-  p(544nm) 


(2.5.9) 


2. 5. 3. 2  Rules-Based  Detector. 


With  the  basic  detector  of  NDSI  or  NDGRI  alone,  objects  with  spectra  similar  to 
skin,  such  as  vegetation,  may  confuse  the  detector.  A  rules-based  detector  from  [28] 
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Figure  11.  Scatterplot  of  the  rules-based  detector  for  living  and  cadaver  measured 
skin  from  [28]  (green)  and  diffuse  modeled  skin  (black)  ranging  from  Type  I/II  to 
Type  V /VI.  The  red  circles  represent  spectral  confuser  data  from  other  objects.  The 
grouping  of  skin  pixels  drive  the  boundary  conditions  for  the  rules-based  detector. 


was  established  to  reduce  false  alarms.  The  rule  is  as  follows: 


= 


1  */  <  l3i,j  <  &2  cind  Cl  <  'fij  <  C2 

0  otherwise 


(2.5.10) 


where  fd  and  7  are  values  computed  per  pixel  in  the  NDSI  and  NDGRI  Equa¬ 
tions  (2.5.8)  and  (2.5.9).  A  pixel  is  considered  skin  if  it  falls  into  the  boundary 
and  rejected  as  another  object  otherwise.  The  boundary  conditions  have  been  set  by 
applying  the  diffuse  skin  model  for  different  types  of  skin  developed  in  [28].  The  mini¬ 
mum  and  maximum  values  for  NDSI  and  NDGRI  associated  with  that  model  are  bi  = 
-0.54079,  62  =  -0.061525,  Ci  =  0.65703,  and  C2  =  0.76779.  In  Figure  11,  a  rules-based 
detector  is  demonstrated  on  diffuse  modeled  skin  for  the  Type  I/II  to  Type  V/VI 
range  represented  by  black  dots,  living  and  cadaver  skin  measurements  from  [28],  and 
confuser  objects  in  red  circles.  The  rules-based  detector  applied  to  an  image  captured 


2-25 


Figure  12.  Image  collected  with  the  Hyper  Spectir  V3  HST3  imager  [20].  (Top)  Gray 
scaled  image  of  the  original  snapshot.  (Bottom)  An  image  mask  with  applied  rule  based 
detector  that  displays  pixels  meeting  criteria  for  skin. 

with  the  Hyper  SpecTIR  V3  hyperspectral  imager  [20]  is  shown  in  Figure  12.  The 
gray  scaled  image  is  above  a  mask  that  shows  pixels  meeting  conditions  for  skin. 

2.6  Summary 

Chapter  2  introduced  material  that  provides  the  foundation  to  all  the  modeling 
and  measurements  we  use  to  characterize  skin.  The  radiometry  section  dehned  all 
the  terms  used  for  the  quantitative  results.  The  human  skin  and  BRDF  sections 
gave  detailed  description  of  the  models  used  for  skin.  The  optics  section  provided  the 
imaging  equations  used  for  the  measurements  systems.  The  hnal  section  described 


2-26 


the  image  processing  techniques  for  skin  detection. 
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3.  Methodology 


In  Chapter  3,  our  method  for  characterizing  surface  reflectance  and  incorporating 
it  into  the  existing  diffuse  skin  reflectance  model  is  shown.  Intermediate  results 
show  our  Bidirectional  Reflectance  Distribution  Function  (BRDF)  measurements  and 
the  £t  to  BRDF  models.  The  remaining  chapter  describes  an  environment  for  running 
the  combined  reflectance  model  on  a  geometry  representing  a  person. 

In  Chapter  2,  we  introduced  the  rules-based  detector  for  determining  skin  pixels 
in  an  image.  The  boundary  conditions  for  NDGRI  and  NDSI  are  set  from  diffuse 
spectral  reflectance  values  for  the  range  of  skin  types  determined  in  [28].  Dependent 
on  the  amount  of  specular  reflection,  the  pixel  may  fall  outside  the  bounds  of  the 
rule  based  detector  and  not  be  considered  skin.  The  remaining  chapter  characterizes 
skin  reflectance  for  changing  illumination  and  detector  conditions  to  improve  the 
rules-based  detector. 

3.1  Combined  Skin  Reflectance  Model 

While  the  human  body  is  uniquely  shaped  and  dependent  on  the  individual  person, 
segments  of  the  body  follow  a  similiar  form,  e.g.,  the  head  is  round,  the  back  of  the 
hand  is  relatively  flat,  and  arms  are  cylindrical.  To  determine  what  is  observed  by 
a  detector  for  a  defined  scene  configuration,  a  combined  skin  reflectance  model  must 
be  applied  to  human  skin  surface  geometries.  Section  3.1  develops  the  combined  skin 
reflectance  model  for  application  to  geometries  that  represent  parts  of  the  human 
body. 
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Figure  13.  A  graphic  of  the  diffuse  plus  specular  reflection  model  used  to  characterize 
human  skin.  The  diffuse  component  is  the  constant  hemispheric  dome  and  the  lobe 
depicts  specular  reflection. 

3.1.1  Combined  Reflectance  Model  Derivation. 

When  characterizing  skin  reflectance,  it  is  best  to  directly  measure  the  skin  re¬ 
sponse  to  light,  accomplished  in  a  later  section.  For  now,  skin  is  considered  neither 
perfectly  diffuse  or  perfectly  specular,  so  a  general  BRDF  model  combines  both  com¬ 
ponents  as: 


PSchlick{,Po{P)  1  ^0)) 


fERDpi^i)  ^r,  A) 


87r(T2(A)cos^6*(6*j,  6*^,  A0)  cos  9i  cos  9r 


+  -  PSchlick{Po{X),m,Or,Acl>)))^,  (3.1.1) 

TT 


where  the  BRDF  is  the  sum  of  specular  (/*)  and  diffuse  {fd)  components  from  Equa¬ 


tions  (2.3.3)  and  (2.3.4),  illustrated  in  Figure  13.  The  angles  /3  and  9  are  defined 
in  Equations  (2.3.5)  and  (2.3.6).  The  Schlick’s  approximation  is  used  as  the  propor¬ 
tionality  term  conserving  energy  between  the  diffuse  and  specular  component.  Both 
the  slope  variance  and  normal  reflectance  terms  are  now  wavelengths  dependent  and 
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Figure  14.  Diagram  of  our  imaging  system  optic,  viewing  a  surface  that  is  rotated  9s 
degrees.  The  surface  reflectance  component  viewed  by  the  imaging  system  is  dependent 
on  the  parameters  shown.  The  general  layout  is  the  same  for  all  of  our  measurement 
scenarios. 

set  for  the  specific  wavelength  of  interest.  The  specular  component  is  represented  by 
surface  reflectance  through  a  BRDF  model  described  in  Section  2.3.  The  diffuse  com¬ 
ponent  is  represented  by  subsurface  reflectance  through  radiometer  measurements. 

Once  a  BRDF  is  established,  the  surface  reflectance  component  viewed  by  our 
imaging  system  can  be  calculated.  The  Directional  Hemispheric  Reflectance  (DHR) 
from  Equation  (2.3.2)  is  a  standard  integration  to  make  that  conversion,  represent¬ 
ing  surface  reflectance  into  the  entire  hemisphere  above  a  media  surface.  However, 
the  DHR  does  not  differentiate  between  locations  of  specular  and  diffuse  reflection 
for  a  discrete  solid  angle  of  a  detector.  To  more  accurately  characterize  combined 
reflectance  observed  by  a  detector,  the  combined  reflectance  is  calculated  over  the 
detector  solid  angle  instead  of  the  entire  hemisphere.  The  diagram  in  Figure  14  il¬ 
lustrates  the  desired  parameters  for  our  modeling  calculations  and  is  similar  to  the 
experimental  scenario  collections.  For  the  surface  reflectance  of  Figure  14,  the  def¬ 
inition  for  BRDF  is  restated  to  solve  for  radiance  from  the  surface,  which  is  then 
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substituted  into  the  equation  for  flux,  Equation  (2.1.6)  as: 


Lr{9i,  9r,  A0,  A)  —  fBRDF{9i,  9r,  A0,  \)Ei{9i,  A),  (3.1.2) 

(3.1.3) 

Q,d  A.S 

Further  modifying  Equation  (3.1.3)  by  dividing  out  incident  flux  and  applying  the 
small  angle  approximation,  leads  to  the  following  form  of  combined  reflectance  ob¬ 
served  by  our  multispectral  system; 


Pcombined{9i,9r,A(j),X)  =  f BRDFi9i,  9r,  A(j),  COS  9r.  (3.1.4) 

Since  the  small  angle  approximation  is  used,  reflectance  over  the  held  of  view  of  the 
detector  is  approximated  constant.  The  small  angle  approximation  reduces  complex¬ 
ity  of  the  model  by  eliminating  the  need  to  integrate  discretely  over  the  surface  area 
where  each  point  on  the  surface  has  a  small  angular  variation  feeding  into  the  BRDF 
model. 

The  subsurface  rehectance  measured  with  our  radiometer  and  modeled  with  the 
Kubelka-Munk  seen  in  [28]  uses  the  assumption  that  light  is  normally  incident  to 
skin  and  the  surface  component  is  Lambertian.  With  the  Lambertian  assumption, 
the  normally  incident  Fresnel  rehection  Equation  (2.2.4)  contributes  a  diffuse  surface 
rehectance  of  0.04  equally  into  the  hemisphere  above  the  skin  surface.  Since  we  are 
considering  incident  light  oh  normal,  we  use  the  DHR  to  determine  trasmittance 
into  the  hrst  layer  of  skin  by  the  law  of  energy  conservation.  The  next  sections  go 
into  detail  on  the  specular  and  dihuse  components,  while  the  following  subsections 
complete  the  discussion  by  describing  an  environment  for  application. 
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3.1.2  Pixel  Field  of  View. 


When  applying  the  small  angle  approximation,  the  area  within  the  held  of  view  of 
the  detector  is  much  smaller  than  the  distance  to  the  detector,  therefore  any  change 
in  location  on  the  projected  pixel  results  in  a  small  change  of  angle  along  the  center 
axis  of  the  optic.  With  a  magnihed  optical  system,  the  detector  projects  using  the 
lens  Equation  (2.2.3).  Each  individual  pixel  for  a  focal  plane  array,  projects  with  the 
same  lens  equation.  The  center  of  the  projected  pixel  is  used  to  calculate  distance  to 
the  lens  and  the  plane  whose  local  normal  is  the  reference  for  incident  and  rehected 
angles  of  light. 

3.1.3  Light  Source. 

In  order  to  calculate  combined  rehectance  as  a  function  of  incident  angle,  the 
incoming  light  rays  are  given  the  same  directionality.  Knowing  the  global  angular 
direction  of  the  incoming  light  enables  us  to  calculate  local  angles  for  individual 
projected  pixels. 

3.1.4  Surface  Modeling. 

Individual  projected  pixels  are  used  to  form  larger  shapes,  therefore  we  hrst  in¬ 
troduce  treatment  of  a  single  projected  pixel.  A  projected  pixel  in  Eigure  15,  is 
represented  in  two  conhgurations  with  respect  to  the  global  reference  axis.  The  left 
image  is  aligned  with  its  normal  along  the  reference  axis,  and  the  right  image  is  ro¬ 
tated  45  degrees.  Eor  incoming  light  at  15  degrees  to  the  right  of  the  global  reference 
axis,  the  left  projected  pixel  incident  angle  is  15  degrees,  while  the  right  projected 
pixel  incident  angle  is  60  degrees  off  the  local  normal.  Similarly,  reflected  angle  to 
a  detector  is  with  respect  to  the  projected  pixel  normal.  The  incident  and  reflected 
angles  for  each  projected  pixel  are  used  for  calculating  combined  reflectance,  therefore 
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(a)  (b) 

Figure  15.  Physical  models  of  a  single  facet  with  respect  to  a  global  reference  axis.  The 
facets  are  directed  with  local  normals,  at  (a)  0  deg  rotation  and  (b)  45  deg  rotation. 


the  most  accurate  geometric  surface  model  provides  the  best  results. 

3. 1.4.1  Cylindrical  Surface  Model. 

As  an  example  consider  a  cylinder  viewed  by  our  imaging  system  with  projected 
pixel  areas  mapped  onto  the  surface  that  are  projected  from  the  focal  plane  array. 
In  Figure  16,  each  neighboring  projected  pixels  in  a  row  has  a  different  incident  and 
reflected  angle,  calculated  with  respect  to  the  local  normal  of  an  individual  projected 
pixel.  Moving  up  or  down  through  the  row  of  projected  pixels  gradually  moves  to 
a  new  out  of  plane  angular  change  to  the  BRDF  calculation.  With  the  small  angle 
approximation,  that  out  of  plane  angular  difference  provides  negligible  change  to 
BRDF.  Further  detail  on  the  geometric  calculation  of  the  specihc  angles  corresponding 
to  each  projected  pixel  is  provided  in  Appendix  A.  While,  simple  shapes  can  only 
provide  an  approximate  geometry,  ultimately  an  accurate  human  geometry  provides 
the  best  surface  representation. 
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Figure  16.  A  cylinder  with  facets  mapped  out  on  its  surface,  projected  from  the  focal 
plane  array  of  our  imaging  system.  Each  facet  has  a  local  normal  used  for  transforming 
the  global  incident  illumination  angle,  into  a  local  angle  used  in  the  calculation  for  skin 
reflectance. 

3.2  Subsurface  Reflectance 

Subsurface  reflectance  is  the  diffuse  component  of  the  combined  reflectance  model, 
i.e.,  Equation  (2.3.3).  In  [28],  and  reviewed  in  the  Chapter  2,  a  nine  layer  Kubelka- 
Munk  model  characterizes  diffuse  reflectance  from  the  subsurface,  based  on  normally 
incident  light.  The  subsurface  contribution  to  reflectance  is  modifled  to  consider  light 
incident  at  angles  off  normal  for  wavelengths  used  in  skin  detection. 

3.2.1  Normal  Illuminated  Reflectance  Measurement. 

An  ASD  FieldSpec3®  radiometer  collects  combined  spectral  reflectance  from  the 
surface  and  subsurface  of  skin  [1] .  The  probe  with  active  source  is  pressed  against  the 
skin  during  measurement  so  the  light  is  normally  incident.  Figure  17  is  a  measurement 
of  Type  I/II  and  Type  V/VI  skin  from  the  held  radiometer  used  for  modeling. 

3.2.2  Subsurface  Reflectance  Model  Fitting. 

The  Type  I/II  and  Type  V/VI  skin  measurements  in  Figure  17  represent  the 
extreme  difference  in  skin  types  that  we  model.  With  the  skin  spectra  from  Figure  17, 
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(a)  (b) 

Figure  17.  Spectral  reflectance  from  normally  incident  illumination  of  (a)  Type  I/II 
and  (b)  Type  V/VI  skin  from  [28].  The  measured  spectra  (blue)  is  collected  with  the 
radiometer  and  modeled  spectra  (green)  is  fitted  useing  Kubelka-Munk. 

the  fitting  parameters  are  found  for  the  Kubelka-Munk  model  in  Section  2.4.2.  The 
measured  spectrum  is  a  sum  of  surface  and  subsurface  components.  The  surface 
component  is  established  in  Chapter  2,  for  the  case  of  normal  incident  light  using  the 
Fresnel  Equation  (2.2.4).  The  subsurface  component  is  calculated  from  the  Kubelka- 
Munk  model  with  parameters  £t  to  match  the  combined  reflectance  of  Figure  17.  With 
the  Root  Mean  Square  Error  (RMSE)  of  Equation  (2.4.14),  a  measure  of  difference 
between  the  measured  and  modeled  data,  we  change  the  htting  parameters  until  we 
minimize  RMSE.  Applying  the  error  function  for  all  combinations  of  the  variable 
parameters  is  computationally  intensive.  In  [28],  a  database  of  radiometer  collected 
data  from  subjects  of  different  skin  types  was  created,  and  a  range  of  the  parameter 
values  was  established  that  best  represented  a  standard  person  for  different  skin  types. 
In  Table  2,  the  fit  parameter  values  are  shown  for  the  Type  I/II  and  Type  V/VI  skin 
spectra  of  Figure  17.  A  similar  htting  is  performed  with  the  same  method  for  any 
person. 
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Surface  Reflectarrce  Surface  Reflectance 

(a)  (b) 

Figure  18.  Subsurface  reflectance  as  a  function  of  surface  reflectance  modeled  with 
Kubelka-Munk  for  (a)  Type  I/II  and  (b)  Type  V/VI  skin.  The  data  represents  wave¬ 
length  regions  of  544nm  (green),  633nm  (dark  blue),  lOSOnm  (red),  and  1580nm  (light 
blue). 


3.2.3  Subsurface  Angular  Dependence. 

With  physical  parameters  ht  to  our  specihc  subjects,  a  model  now  exists  for  nor¬ 
mal  incident  illumination.  Due  to  the  conservation  of  energy,  as  front  surface  re¬ 
flectance  increases,  the  amount  of  flux  transmitted  into  the  first  layer.  Front  surface 
reflectance  (Rq)  is  calculated  as  Schlick  reflectance  (pschUck)  from  Equation  (2.2.7), 
with  the  value  subtracted  from  1  to  And  the  transmittance  (Tq).  As  the  incident  angle 
changes  for  a  specified  projected  pixel,  a  new  front  surface  reflectance  is  calculated, 
thereby  introducing  angular  dependence  on  the  subsurface  component  of  the  BRDF 
model.  With  the  established  fitting  parameters  for  the  normally  incident  light  in  the 

Table  2.  Kubelka-Munk  fitting  parameters  for  our  Type  I/II  and  Type  V/VI  skin 
measurements. 


Parameter 

Typel/II 

Type  V/VI 

M elanosome  % 

3.5 

40 

Blood  Level  amount 

0.9 

1.1 

Oxygenation  % 

50 

18 

Subcutaneous  Scale 

0.46 

0.72 

3-9 


previous  subsection,  the  Kubelka-Munk  model  is  run  while  varying  Rq  from  0.04  to 
1.  In  Figure  18,  pd  is  plotted  as  a  function  of  Rq  for  the  Type  I/II  and  Type  V/VI 
skin  spectra  of  Figure  17.  A  polynomial  htting  is  performed  for  each  of  the  four 
wavelengths  so  that  the  Knbelka-Mnnk  model  does  not  need  to  be  rerun  for  each 
projected  pixel  in  a  model  of  a  particular  person. 

3.3  Surface  Reflectance 

Surface  reflectance  observed  by  the  detector  is  the  specular  component  of  the 
combined  reflectance  model,  i.e.,  Eqnation  (2.3.4).  In  [28],  the  lambertian  assnmp- 
tion  was  nsed  to  approximate  skin  as  a  diffuse  surface  in  order  to  simplify  the  model 
for  combined  reflectance.  We  introdnce  a  method  to  replace  the  lambertian  assnmp- 
tion  with  a  more  representative  model  that  acconnts  for  the  dependency  of  combined 
reflectance  on  sonrce  and  detector  angular  locations.  BRDF  is  the  tool  that  char¬ 
acterizes  a  medinm  with  those  angnlar  dependencies  to  generate  a  more  accnrate 
model.  While  there  are  many  BRDF  models  that  have  been  created  for  specihc  ap¬ 
plications,  we  focns  on  a  Gaussian  microfacet  model  dne  to  a  similiar  application 
in  the  simnlation  tool  discussed  later.  The  BRDFs  for  the  visible  wavelengths  are 
measnred  directly  with  a  commercial  system,  limitations  force  the  BRDF’s  for  the 
NIR  wavelengths  to  be  derived  from  image  measurements. 

3.3.1  BRDF  ELM. 

When  working  with  surfaces  dehned  with  a  BRDF,  reflectance  may  change  with 
sonrce  and  detector  conhgnration.  While  prior  skin  detection  work  focussed  on  sonrce 
and  camera  positions  normal  to  the  sample  snrface,  the  work  that  follows  focuses  on 
off  normal  measnrement  and  modeling.  The  DHR  valne  for  the  reference  Spectralon® 
is  an  adeqnate  diffuse  reflectance  valne  to  use  in  ELM  while  normal  to  the  snrface. 
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but  it  breaks  down  as  you  move  the  source  off  normal.  While  Spectralon®  is  often 
considered  a  standard  diffuse  material,  it  has  been  shown  in  [7]  that  it  gets  increasingly 
more  specular  as  the  illumination  incident  angle  moves  off  normal  (see  Appendix  B  for 
Spectralon®  BRDF).  By  substituting  BRDF  from  Equation  (3.1.4)  in  for  reflectance 
in  the  ELM  Equations  2.5.5  and  2.5.6,  the  constant  associated  with  the  camera  solid 
angle  is  lumped  into  the  a(X )  parameter  to  create  the  following  form  for  ELM: 


Leifii)  Or,  A^,  A)  —  a(A)  ■  fBRDpiOi,  Or,  A(f),  A)  +  5(A),  (3.3.1) 


a(A) 


_ L2(A)-Li(A) _ 

fBRDF,2{0i,  Or,  A0,  A)  —  f BRDF,l{0i,  Or,  A(j),  A)  ’ 


5(A) 


-hl(A)  ■  fBRDF,2{0i,  Or,  A(j),  A)  —  L2(A)  '  fBRDF,l{0i,  Or,  A(f),  A) 
fBRDF,2{0i,  Or,  A(j),  A)  —  fBRDF,l{0i,  Or,  A(j),  A) 


(3.3.2) 

(3.3.3) 


where  fpROF,!  and  fBRDF,2  are  the  BRDE  of  our  light  and  dark  reference  panels. 


3.3.2  BRDF  Measurement. 

Our  preferred  approach  to  characterizing  the  BRDF  of  skin  is  with  direct  mea¬ 
surement.  The  Complete  Angle  Scatter  Instrument  (CAST®)  is  a  commercially  made 
scattero  meter  used  to  measure  BRDF,  seen  in  Figure  19.  In  our  research,  the  CAS  I® 
system  measures  visible  (VIS)  wavelengths,  but  is  not  setup  for  our  near-infrared 
(NIR)  wavelengths.  Since  we  cannot  measure  the  NIR  with  the  CASI®,  a  different 
approach  is  followed  to  ht  BRDF  parameters  from  images. 

3.3. 2.1  CASI®  Scatterometer  VIS  Measurements. 

The  CASI®  is  a  BRDF  measurement  device  manufactured  by  Schmitt  Measure¬ 
ment  Systems.  With  a  similar  description  to  the  Gonioreflectometer  reviewed  in 
Chapter  2,  an  armatnre  holding  the  detector  travels  around  the  mounted  sample 
gathering  unpolarized  data  for  a  fixed  incident  angle  at  a  set  wavelength.  The  method 
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Figure  19.  The  CASI  scatterometer  featuring  a  detector  and  adjustable  sample  mount. 
The  detector  is  attached  to  an  arm  that  rotates  around  the  sample.  The  green  arrow 
indicates  the  path  of  light  from  a  laser  source,  onto  the  sample  which  then  reflects  with 
the  BRDF  mapped  out  by  the  rotating  detector. 


for  collection  and  plotted  resnlts  of  skin  BRDF  is  presented. 

The  BRDF  measnrements  are  collected  with  the  CASI®  at  onr  VIS  wavelengths. 
The  back  of  a  Type  I/II  hand  is  placed  at  the  sample  location  and  in-plane  BRDF  is 
collected  for  the  set  incident  angles  of  0,  30,  and  60  degrees.  The  corresponding  plots 
are  shown  in  Figure  20  for  both  VIS  wavelengths  used  in  detection.  The  dip  for  the  0 
deg  incidence  plot,  is  due  to  the  detector  blocking  the  source  during  those  collections 
interupting  detection  for  those  few  points.  For  the  30  degree  incidence  plot  we  see 
noise  in  the  collection,  which  may  have  been  caused  by  movement  of  the  hand  during 
the  collection.  Fitting  to  data  with  this  type  of  noise  may  pull  us  from  an  accurate 
htting,  and  so  we  move  forward  to  htting  our  BRDF  model  with  the  0  and  60  degree 
plots  in  the  next  section.  For  the  NIR  wavelengths  an  image-based  approach  was 
applied  shown  in  Appendix  D.  Due  to  technical  difficulties  related  to  non  constant 
illumination  and  reference  materials  this  method  was  not  used.  Without  a  direct 
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(a)  (b) 


(c) 

Figure  20.  CASI®  scatterometer  measured  BRDF  at  544nm  (green)  and  633nm  (blue) 
of  Type  I/II  skin  for  incident  angles  of  (a)  0  deg,  (b)  30  deg,  and  (c)  60  deg. 


BRDF  measurement,  the  fitting  parameters  of  our  BRDF  model  in  the  NIR  are  £t 
directly  to  images  collected  with  the  AFIT  SERG  multispectral  system  in  the  next 
subsection. 

3.3.3  Specular  Reflection  Model. 

Since  it  would  take  too  long  to  measure  the  BRDF  of  skin  for  every  incident  angle, 
including  in  and  out  of  plane  measurements,  a  model  is  fit  to  data  collected  for  a  few 
incident  angles.  We  present  our  BRDF  modeling  of  skin  for  the  VIS  wavelengths  and 
then  fit  our  BRDF  model  to  images  collected  with  the  AFIT  SERG  system  for  the 
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Figure  21.  Type  I/II  skin  measurements  (red)  and  corresponding  fitting  of  the  micro¬ 
facet  BRDF  model  (blue)  for  (top)  544nm  and  (bottom)  633nm.  The  incident  angles 
are  set  to  (left)  0  deg  and  (right)  60  deg. 


NIR  wavelengths. 

3.3.3. 1  VIS  BRDF  Model  Parameter  Fitting. 

The  Gaussian  nature  of  the  specular  lobe  of  media  has  led  to  the  standard  practice 
of  using  a  Gaussian  distribution  for  microfacet  BRDF  modeling.  The  microfacet 
model,  while  dependent  on  the  interface  indices,  is  also  dependent  on  the  orientation 
distribution  of  individual  microfacets.  The  basis  for  generating  an  accurate  model  is 
by  assigning  a  Gaussian  distribution  to  the  orientation,  and  fitting  the  parameters  to 
measured  data.  The  two  htting  parameters  for  the  Gaussian  model  are  and  po- 
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Figure  22.  RGB  image  of  the  Type  I/II  subject,  taken  with  the  SERG  multispectral 
detection  system.  The  subject  is  in  the  center  with  reference  panels  on  both  sides  to 
transform  the  scene  into  reflectance  space. 

With  in-plane  data  collected,  the  two  parameters  are  fit  by  minimizing  the  root  mean 
square  error  (RMSE)  between  the  model  and  measured  values.  In  Figure  21,  the 
model  is  £t  to  the  CASI  measured  data  for  the  544nm  and  633nm  wavelength  region. 
It  was  determined  the  best  fitting  for  544nm  is  a  variance  of  0.0313  and  normal  Schlick 
coehcient  of  0.03  with  an  approximate  RMSE  of  1.8%.  The  best  htting  for  633nm  is 
a  variance  of  0.0292  and  Schlick  coefficient  of  0.038  with  an  approximate  RMSE  of 
8.2%.  Applying  Equation  (2.2.4),  we  can  back  out  the  approximated  spectral  index 
at  our  detection  wavelengths  of  544nm  (n2  =  1.42)  and  633nm  (77-2  =  1.42). 

3. 3. 3. 2  NIR  BRDF  Model  Parameter  Fitting. 

Without  a  direct  way  to  measure  the  BRDF  at  the  NIR  wavelengths,  the  BRDF 
modeling  parameters,  cr^  and  po,  are  approximated  with  an  image  collected  by  the 
AFIT  SERG  detection  system.  In  Figure  22,  the  RGB  image  from  the  AFIT  SERG 
system  of  a  Type  I/II  subject,  illuminated  10  degrees  off  normal  is  shown.  The  image 
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Figure  23.  Normalized  plot  across  the  Type  I/II  subject  forehead  of  Figure  22.  The 
lines  represent  the  different  wavelengths  for  the  AFIT  SERG  system  of  544nm  (dark 
blue)  ,  633nm  (green),  lOSOnm  (red),  and  1580nm  (light  blue). 

is  then  converted  into  reflectance  space  using  the  reference  panels  with  ELM.  Since  the 
cameras  have  been  aligned  so  that  the  images  would  correspond  with  each  other  for  the 
detection  algorithms,  NDSI  and  NDGRI,  a  row  of  pixels  across  the  forehead  is  selected 
for  analysis.  In  Figure  23,  the  normalized  spatial  plot  across  the  subject  forehead 
is  shown  for  each  wavelength,  where  the  normalized  plot  represents  the  change  of 
reflectance  in  a  common  scale  comparison.  Since  these  lines  represent  the  same  surface 
geometry,  it  is  our  assumption  that  the  difference  in  slope  is  a  difference  in  BRDF 
affecting  surface  and  subsurface  reflectance  observed  by  the  detection  system.  Since, 
we  have  already  fit  the  BRDF  parameters  for  544nm  and  633nm,  the  incident  {6i) 
and  reflection  {6r)  angles  that  define  the  surface  geometry  for  the  pixel  in  the  row 
across  the  forehead  are  solved  with  the  following  equations: 


Pj(544nm)  =  pd(544nm,  6*,)  +  psi^AAnm,  9i,  9r), 


(3.3.4) 
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(a)  (b) 


(c)  (d) 

Figure  24.  Modeled  (blue)  compared  with  measured  (green)  reflectance  across  the  Type 
I/II  subjects  forehead  from  an  image  collected  with  the  AFIT  SERG  multispectral 
camera  system.  The  fitting  is  based  on  the  surface  geometry  of  the  forehead.  The 
plots  correspond  to  wavelength  regions  of  (a)  544nm,  (b)  633nm,  (c)1080nm,  and  (d) 
1580nm. 


Pj{633nm)  =  pd(633nm,  6i)  +  ps{633nm,  9i,  9r),  (3.3.5) 

where  pd  is  observed  subsurface  reflectance  modeled  with  Kubelka-Munk  and  ps  is 
observed  surface  reflectance  modeled  with  the  small  angle  BRDF  Equation  (3.1.4). 
The  same  surface  geometry  is  run  for  lOSOnm  and  1580nm,  in  order  to  determine 
a  best  htting  for  the  BRDF  parameters,  with  results  from  £t  parameters  seen  in 
Figure  24.  The  best  htting  for  lOSOnm  is  a  variance  of  0.0214  and  normal  Schlick 
coehcient  of  0.045  with  an  approximate  RMSE  of  7.3%.  The  best  fitting  for  1580nm 
is  a  variance  of  0.0146  and  Schlick  coefficient  of  0.042  with  an  approximate  RMSE  of 
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Figure  25.  AFIT  SERG  multispectral  detection  system  constructed  in  [30]  and  used 
for  our  experimental  measurements.  The  system  consists  of  an  RGB  camera  and  2 
Goodrich  SIU  cameras  covering  the  range  of  400-1700nm. 

6.1%.  Applying  Equation  (2.2.4),  we  can  back  out  the  approximated  spectral  index 
at  our  detection  wavelengths  of  lOSOnm  (n2  =  1.54)  and  1580nm  (n2  =  1.51). 

3.4  Experimental  Scenario  Measurement  and  Simulation 

In  order  to  verify  our  modeling  results,  images  are  collected  with  our  AFIT  SERG 
multispectral  system,  seen  in  Figure  25,  for  a  controlled  indoor  experimental  scenario. 
The  study  in  [30]  created  the  AFIT  SERG  multispectral  system  for  skin  detection, 
and  compared  collected  data  to  results  from  the  diffuse  skin  reflectance  model  in 
[28].  We  perform  a  similar  comparison  of  collected  data,  but  we  use  modeling  and 
simulation  results  accounting  for  combined  reflectance  that  changes  with  the  scene 
conhguration. 

In  our  experimental  scenario,  we  limit  the  conhguration  changes  to  relocating  the 
light  source.  Data  is  collected  for  illumination  angles  of  10,  30,  60  and  85  degrees 
from  normal  of  the  human  subject.  The  detector  is  located  normal  to  the  subject 
and  at  a  distance  of  31  ft.  The  human  subject  is  of  Type  I/II  skin  and  remains 
in  the  same  position  for  each  illumination  angle  collected.  Any  movement  of  the 
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camera  or  positioning  of  the  human  subject  would  introduce  additional  variables  into 
the  collection.  While  we  discuss  future  work  associated  with  variables  to  the  human 
subject  in  Chapter  5,  the  next  section  introduces  an  environment  to  simulate  scenarios 
with  a  variety  of  different  conhgurations. 

3.4.1  Scenario  Simulation. 

While  basic  shapes  can  represent  parts  of  the  human  body,  modeling  with  the 
actual  form  of  a  person  provides  the  most  accurate  geometry.  With  the  drive  of 
the  motion  picture  industry  to  constantly  improve  upon  its  animations  and  impress 
its  audience,  powerful  software  applications  for  rendering  realistic  scenes  have  been 
created.  Blender®  is  a  software  package  dedicated  to  providing  realism  to  three  di¬ 
mensional  animations  [2].  Within  Blender®,  a  virtual  object  is  built  or  loaded  in 
three  dimensional  space  and  rendered.  We  focus  on  simulating  the  indoor  experi¬ 
mental  scenario  collected  with  the  AFIT  SERG  multispectral  system  for  the  same 
conhgurations. 

Within  the  Blender®  interface,  a  model  is  built  however  the  user  chooses,  but 
constructing  an  exact  three  dimensional  replica  of  a  specihc  person  would  be  diffi¬ 
cult.  Blender®  features  a  method  to  import  Wavefront  object  hies  that  contain  a 
three  dimensional  model.  With  access  to  the  three  dimensional  surface  scanner  from 
the  711th  Human  Performance  Wing  (AFRL/RHPA)  at  Wright-Patterson  AFB,  the 
human  subject  is  digitized.  In  Figure  26,  an  illustration  of  the  digitized  subject  is 
loaded  into  Blender®  for  simulations. 

With  the  three  dimensional  subject  loaded  into  Blender®,  the  surface  rehectance 
characteristics  are  set  to  match  that  of  skin.  The  Ward  model  settings  within  the 
Blender®  BRDF  toolbox  allow  us  to  set  the  two  Gaussian  parameters  we  ht  from 
our  BRDF  measurements.  A  rendering  is  done  for  each  wavelength  represented  by 
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Figure  26.  The  digitized  three  dimensional  human  subject  located  between  two  ref¬ 
erence  pannels  in  Blender®.  The  panels  are  assigned  lambertian  reflectance  values  of 
0.97  for  the  light  colored  panel  and  0.03  for  the  dark  colored  panel.  The  human  subject 
is  assigned  reflectance  characteristics  with  the  toolset  of  diffuse  and  specular  shaders. 

its  own  Gaussian  parameters. 

In  the  Blender®  environment,  detector  and  source  characteristics  are  dehned  to 
be  similar  to  those  in  the  experimental  scenario.  In  Figure  27,  multiple  camera  and 
light  source  locations  are  illustrated  and  two  reference  panels  are  placed  on  the  side  of 
the  scanned  subject,  similar  to  the  measurements.  All  objects,  including  the  digitized 
subject,  can  be  repositioned  anywhere  in  the  virtual  environment  for  future  scenarios. 

The  output  to  the  simulated  scene  is  a  rendered  image  in  digital  space  that  de¬ 
scribes  a  pixel  with  an  intensity  value.  The  two  panels  are  dehned  in  Blender®  as 
diffuse  surfaces  with  different  rehectance  values  of  0.97  and  0.03.  While  the  scene  is 
illuminated,  these  panels  act  as  reference  materials  for  performing  ELM.  Applying 
ELM  to  the  reference  panels  of  known  diffuse  rehectance  creates  a  method  for  con¬ 
verting  the  entire  image  from  digital  space  to  rehectance  space.  A  simple  example 
is  demonstrated  in  Appendix  C  with  a  dihuse  object  in  place  of  the  BRDF-dehned 
person. 
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Figure  27.  Blender®  interface  with  2  cameras  and  2  light  sources,  demonstrating  that 
each  can  be  placed  anywhere  in  the  three  dimensional  space.  The  human  subject  and 
reference  materials  are  the  in-scene  objects  that  are  rendered. 


3.5  Summary 

In  Chapter  3,  BRDF  models  were  fit  for  the  wavelengths  used  in  detection.  A 
method  described  integration  of  the  BRDF  model  into  the  existing  diffuse  reflectance 
model.  In  the  next  chapter,  results  to  the  surface  models  are  compared  with  measured 
data  from  the  AFIT  SERG  multispectral  system. 
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4.  Results  and  Analysis 


In  Chapter  4,  our  experimental  measurement  and  human  surface  modeling  results 
are  presented.  The  effect  of  off  normal  angular  illumination  on  the  rules-based 
detector  is  shown.  Finally,  a  metric  is  used  for  the  correlation  between  measured  and 
modeled  results. 

In  Chapter  3,  we  introduced  the  dependencies  of  the  BRDF  model,  illustrated  in 
Figure  28.  Skin  front  surface  reflectance  calculated  with  Fresnel  equations  in  Figure  2, 
demonstrates  that  for  illumination  angles  from  0  to  near  60  degrees,  the  front  surface 
reflectance  of  skin  is  approximately  0.04.  This  low  variation  for  these  angles  motivated 
the  Lambertion  assumption  for  front  surface  reflectance  in  the  existing  diffuse  model 
[28].  However,  for  illumination  angles  greater  than  60  degrees,  the  front  surface 
reflectance  increases  and  is  more  of  a  distribution,  thereby  motivating  us  to  shift 
away  from  the  lambertian  skin  assumtion.  While  specularity  affects  some  pixels  with 
a  large  increase  in  reflectance  when  viewing  into  the  specular  lobe,  other  pixels  will 
have  a  large  decrease  in  reflectance  when  viewing  outside  the  specular  lobe.  Figure  28 
illustrates  that  the  location  we  view  into  the  BRDF  has  a  major  impact  on  apparent 
reflectance.  We  show  the  effect  in  resulting  rules-based  detector  scatter  plots.  With 
BRDF  modeling,  a  method  now  exists  to  simulate  many  different  scenarios  in  order 
to  determine  adjustable  boundary  conditions  for  the  rules-based  detector,  dependent 
on  the  scene  conhguration. 

4.1  Measurements  Results 

Images  from  the  Air  Force  Institute  of  Technology  (AFIT)  Sensors  Exploitation 
Research  Group  (SERG)  multispectral  system  are  presented,  demontrating  off  normal 
illumination  effects  on  apparent  reflectance  and  the  rule  based  detector.  The  resulting 
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Figure  28.  Diagram  from  [19],  demonstrating  the  effect  we  characterize  in  the  human 
skin  results.  There  are  two  facet  in  the  diagram,  the  left  one  is  normal  to  the  observer 
and  the  right  one  is  rotated  25  degrees  towards  the  illumination  source.  The  facet 
BRDF  drawn  with  the  thick  orange  line,  changes  as  a  function  of  incident  angle  affecting 
reflectance  seen  by  the  observer.  The  dashed  blue  line  goes  from  the  observer  to  the 
facet  and  the  blue  dot  intersects  the  applicable  point  on  the  BRDF  curve. 


Table  3.  Directional  Hemispheric  Reflectance  Values  for  our  light  and  dark  Spectralon 
Reference  Panels. 


Wavelength 

White  Panel 

Dark  Panel 

544?7,m 

0.989 

0.075 

633nm 

0.990 

0.079 

lOSOnm 

0.992 

0.104 

ISSOnm 

0.988 

0.133 

histograms  serves  as  a  comparison  to  modeled  data. 


4.1.1  Measurement  Uncertainty  Analysis. 

When  measuring  reflectance  from  off-normal  illumination,  our  reference  materials 
exhibit  some  form  of  specular  reflection  changing  with  wavelength,  since  they  are 
not  perfectly  diffuse  (seen  in  Appendix  B).  With  our  reference  Spectralon®  panels, 
Labsphere  provided  us  with  normal  illuminated  DHR  values  for  wavelengths  in  the 
visible  (VIS)  to  near-infrared  (NIR).  The  DHR  values  for  our  wavelengths  of  interest 
are  listed  in  Table  3.  Since  we  do  not  have  the  BRDF  of  our  panels,  we  do  not  know 
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how  the  apparent  reflectance  of  our  reference  materials  changes  with  illumination 
angle.  Since  we  use  the  panels  in  a  linear  regression  for  the  Empirical  Line  Method 
(ELM),  uncertainty  in  the  panel  reflectance  at  off  normal  illumination  angles  intro¬ 
duces  uncertainty  in  our  computed  reflectance.  Our  simulated  Blender®  environment 
uses  ideal  reference  materials;  therefore,  the  uncertainty  is  a  cause  of  difference  be¬ 
tween  the  measured,  simulated,  and  true  reflectances.  To  calculate  the  uncertainty, 
we  use  the  BRDF  Spectralon®  plots  (Appendix  B)  at  633nm  for  VIS  wavelengths 
and  3390nm  as  the  closest  available  for  the  NIR  wavelengths.  In  Chapter  3,  with 
the  small  angle  assumption,  we  relate  apparent  reflectance  to  BRDF  through  a  solid 
angle  that  remains  constant  for  our  measurements.  Therefore,  we  relate  a  change  in 
BRDF  to  a  change  in  apparent  reflectance  of  the  pannels.  The  largest  error  across 
the  BRDF  plots  shown  in  Appendix  B  is  seen  at  75  deg  incident  illumination  for  both 
wavelengths.  We  calculate  the  change  in  BRDF  from  0  to  75  deg  illumination  angle 
for  a  camera  position  of  0  deg,  as  a  comparable  error  to  our  DHR  values.  Therefore, 
using  the  normal  incident  DHR  values  at  illumination  angles  off  normal  introduces  a 
maximum  estimated  error  of  4.1%  in  the  VIS  and  7.4%  in  the  NIR. 

4.1.2  Experimental  Scenario  Measurement. 

In  Figure  29,  each  image  for  the  scenario  shown  configured  as  Figure  30,  represent¬ 
ing  each  wavelength  for  detection  of  the  Type  I/II  subject,  is  mapped  into  reflectance 
space.  The  longer  wavelength  colors  (i.e.,  red)  correspond  to  high  reflectance  and 
shorter  wavelength  colors  (i.e.,  blue)  correspond  to  low  reflectance  following  the  color 
scale.  Since  the  source  is  only  10  degrees  off  normal  for  these  images,  a  majority  of  the 
pixels  are  viewing  skin  that  have  a  local  incident  illumination  angle  under  60  degrees. 
However,  locations  of  high  specular  reflection  directed  into  the  camera  contribute  to 
the  high  reflectance  values  and  locations  of  high  specular  reflection  directed  away 
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(c)  (d) 

Figure  29.  Image  taken  with  the  AFIT  SERG  multispectral  system,  seperated  into 
wavelength  regions  of  (a)  544nm,  (b)  633nm,  (c)  lOSOnm,  and  (d)  1580nm.  The  scenario 
has  the  camera  normal  to  the  subject  and  the  light  source  10  deg  off  normal.  The  images 
are  in  reflectance  space  differentiated  in  value  by  the  corresponding  color  scale. 


from  the  camera  contribute  to  lower  reflectance  values. 

To  demonstrate  the  effect  specular  reflection  has  on  our  measurements,  images  are 
also  aquired  with  the  AFIT  SERG  multispectral  system  for  illumination  angles  of  30, 
60,  and  85  degrees  off  normal.  As  the  illumination  angle  moves  further  off  normal  to 
the  subject,  shown  in  Figure  31  for  60  degrees  illumination,  pixel  reflectance  values 
spread  out  above  and  below  the  normal  incident  diffuse  value.  The  effect  is  explained 
by  the  notion  that  as  we  are  close  to  normal  illumination,  seen  with  the  10  degree 
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Figure  30.  Diagram  of  our  experimental  scenario  for  our  AFIT  SERG  multispectral 
system  viewing  a  Type  I/II  peson.  The  source  is  located  9  degrees  from  normal  to  the 
person. 

images,  a  majority  of  the  light  is  incident  on  skin  under  local  illumination  of  60 
degrees  for  a  front  surface  reflectance  of  0.04  resulting  in  an  apparent  reflectance 
similar  to  the  radiometer  measured  value  for  those  pixels.  For  those  conditions, 
subsurface  reflectance  is  the  driver  for  observed  reflectance.  As  we  move  the  angular 
illumination  towards  85  degrees,  more  of  those  pixels  are  viewing  reflectance  from  skin 
local  illuminated  greater  than  60  degrees.  According  to  Fresnel,  the  rate  of  change 
on  reflectance  for  local  incident  angles  greater  than  60  degrees  is  steep.  The  sharp 
slope  combined  with  the  random  surface  geometry  on  the  subject  results  in  values 
across  the  reflectance  scale  for  large  incident  angles.  The  side  of  the  subject  with  the 
illumination  source  exhibits  an  increase  in  reflectance  since  the  specular  reflection  is 
directed  to  the  camera.  Since  the  illumination  on  the  side  of  the  face  without  the  light 
source  is  not  constant,  reflectance  is  lowered.  This  is  an  artifact  of  the  measurement 
that  is  characterized. 
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(c)  (d) 

Figure  31.  Images  captured  with  the  AFIT  SERG  multispectral  system  illuminated  60 
degrees  off  normal  to  the  subject  and  converted  into  reflectance  space  for  wavelengths 
of  (a)  544nm,  (b)  633nm,  (c)  lOSOnm,  and  (d)  1580nm. 

The  measurements  are  used  to  find  NDGRI  and  NDSI  values  for  each  pixel  result¬ 
ing  in  a  scatterplot  for  the  rule  based  detector  in  Figure  32.  While  all  dots  plotted 
are  skin  pixels  the  blue  dots  represent  pixels  detected  as  skin,  while  the  green  dots 
are  rejected  and  are  considered  as  other  objects.  The  plots  show  how  pixels  fall 
outside  the  boundary  for  skin  detection  as  the  illumination  incident  angle  increases. 
At  85  degree  illumination,  nearly  every  pixel  is  rejected,  since  they  fall  outside  the 
boundary.  A  trend  in  the  shift  of  pixels,  for  the  respective  wavelength,  from  the 
diffuse  value  measured  with  the  radiometer  to  high  or  low  reflectance  values  depends 
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(c)  (d) 

Figure  32.  Rules-based  scatterplot  of  the  data  collected  with  the  AFIT  SERG  system 
for  illumination  angles  of  (a)  10  deg,  (b)  30  deg,  (c)  60  deg,  and  (d)  85  deg.  The  blue 
points  indicate  skin  that  meets  the  rule  based  criteria;  the  green  points  are  pixels  that 
are  skin,  but  do  not  meet  the  critera  for  the  detector  based  on  boundary  set  from  the 
diffuse  model. 


on  scene  configuration.  Accurate  characterization  of  the  changing  pixels  is  an  indica¬ 
tor  for  adjusting  the  rules-based  boundary  conditions.  Figure  33  is  a  histogram  for 
the  distribution  of  pixels  in  the  hrst  quadrant  of  the  scatterplot  and  are  used  for  a 
correlation  comparison  in  the  next  section. 

4.2  Modeling  Results 

Reflectance  modeling  of  surfaces  viewed  from  the  perspective  of  the  AFIT  SERG 
multispectral  detection  system  is  explored.  The  conhguration  used  for  our  experi- 
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Figure  33.  Histogram  of  the  rules-based  scatterplot  of  the  data  collected  with  the  AFIT 
SERG  system  for  illumination  angles  of  (a)  10  deg,  (b)  30  deg,  (c)  60  deg,  and  (d)  85 
deg.  The  graphic  represents  the  distribution  of  pixels  falling  into  a  specific  NDSI  vs 
NSGRI  bin. 


mental  measurements  is  used  within  the  modeling  environment  for  comparison. 

4.2.1  Human  Surface  Simulation. 

Simulation  with  the  most  accurate  three  dimensional  model  is  our  approach  to 
estimating  apparent  reflectance.  In  order  to  simulate  the  experimental  scenario, 
Blender®  is  utilized  with  the  three  dimensional  person.  In  this  section  we  give  simu¬ 
lated  results  to  the  experimental  scenarios  with  the  digitized  human  subject. 

The  subsurface  spectral  reflectance  that  is  driven  by  Kubelka-Munk  modeling  is 
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Figure  34.  Results  of  diffuse  only  skin  modeling  for  incident  illumination  angles  of  (a) 
10  deg,  (b)  30  deg,  (c)  60  deg,  and  (d)  85  deg. 


not  included  in  the  Blender®  package.  Objects  illuminated  within  Blender®  can  have 
an  underlying  diffuse  reffectance  that  is  adjustable.  The  diffuse  reffectance  is  set  to 
the  radiometer  measured  reffectance  value  unique  for  the  human  subject  Type  I/II 
skin  and  for  the  respective  wavelength  rendered.  The  software  has  a  front  surface 
proportionality  correcting  the  diffuse  value  with  a  change  in  local  illumination  an¬ 
gle.  The  BRDF  parameters  used  for  surface  roughness  ct^(A)  and  normal  illuminated 
reffectance  Po(A)  are  both  spectral  terms  adjusted  with  a  change  in  wavelength. 

A  plot  of  our  rules  based  detector  on  data  modeled  with  a  diffuse  only  compo¬ 
nent  is  presented  in  Figure  34.  The  results  show  that  our  boundary  conditions  are 
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(c)  (d) 

Figure  35.  Blender  rendered  images  converted  to  reflectance  space  modeling  (a)  544nm, 
(b)  633nm,  (c)  lOSOnm,  and  (d)  1580nm.  The  modeling  scenario  has  the  camera  normal 
to  the  human  subject  and  source  10  degrees  off  normal. 

a  good  indicator  for  skin  when  only  considering  the  diffuse  reflectance  component. 
Since  NDGRI  is  a  measure  of  difference  between  two  wavelengths,  the  change  is  front 
surface  reflectance  proportionally  changes  both  544nm  and  633nm  reflectance  while 
preserving  the  difference  between  them.  For  NDSI  our  1580nm  wavelength  has  low  re¬ 
flectance  and  the  change  is  front  surface  reflectance  is  not  proportional  with  lOSOnm, 
resulting  in  a  shift  of  the  cluster  within  the  scatterplot. 

The  scenario  where  the  source  is  10  degrees  off  normal  to  the  human  subject  is 
rendered  with  images  shown  in  Figure  35.  It  is  evident  that  variations  across  the 
face  are  related  to  the  specular  reflectance  and  they  match  well  to  the  corresponding 
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(c)  (d) 

Figure  36.  Rendered  images  for  a  scene  configuration  illuminated  60  degrees  off  normal 
to  the  subject  and  converted  into  reflectance  space  for  wavelengths  of  (a)  544nm,  (b) 
633nm,  (c)  lOSOnm,  and  (d)  1580nm. 


measured  images  from  Figure  29.  The  scenario  where  the  source  is  60  degrees  off 
normal  to  the  human  subject  is  rendered  with  images  shown  in  Figure  36.  Within 
the  60  deg  illuminated  images  the  nonconstant  illumination  on  the  other  side  of  the 
face  is  similiar  to  that  of  the  measurements.  Although  nonconstant  illumination  is 
not  ideal  conditions  for  performing  the  emperical  line  method,  a  reduced  apparent 
reflectance  is  shown  to  be  similar  to  the  measurements. 

A  metric  for  the  measure  of  overlap  between  the  modeling  and  measurement  his- 
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Figure  37.  Rules-based  scatterplot  of  the  data  collected  with  the  AFIT  SERG  system 
and  Blender  simulated  data,  for  illumination  angles  of  (a)  10  deg,  (b)  30  deg,  (c)  60  deg, 
and  (d)  85  deg.  The  blue  points  indicate  skin  that  meets  the  rules-based  criteria;  the 
green  points  are  skin  pixels  not  meeting  the  critera  for  skin  based  on  diffuse  modeling. 
The  red  circles  are  blender  simulated  data.  The  rectangle  is  the  boundary  for  skin 
pixels  in  the  rules-based  detector. 


tograms  is  the  Bhattacharyya  coefficient  (BC)  calculated  as: 

n 

=  (4-2.1) 

i=l 

where  Pi  and  qi  represent  the  percent  of  the  cumulative  number  of  pixels  in  the  bin 
for  measured  and  modeled  histograms.  A  BC  closer  to  1  indicates  higher  correlation  in 
the  data,  where  a  BC  closer  to  0  indicates  less  correlation  in  the  data.  In  Figure  37, 
the  rules-based  detector  is  applied  to  the  Blender®  data  in  red  and  plotted  over 
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Figure  38.  Histogram  of  the  rules-based  scatterplot  of  the  blender  modeled  scenario 
with  illumination  angles  of  (a)  10  deg,  (b)  30  deg,  (c)  60  deg,  and  (d)  85  deg.  The 
graphic  represents  the  distribution  of  pixels  falling  into  a  specific  NDSI  vs  NSGRI  bin. 


the  scenario  measured  data  points.  The  rendered  data  aligns  favorably  with  the 
measured  data  from  the  scatterplot.  When  comparing  the  diffuse  only  model  with 
the  diffuse  plus  specular  model,  it  is  clear  that  an  increase  in  specular  reflectance 
provides  the  general  shift  in  pixels  out  of  the  boundary  region.  It  is  evident  that 
modeling  with  a  diffuse  plus  specular  model  gives  a  better  reflectance  estimate  than 
just  the  diffuse  model  alone.  The  added  specular  component  allows  us  to  estimate  the 
shift  in  the  cluster  of  pixels  representing  skin  when  applying  the  rules-based  detector. 
The  corresponding  distribution  of  values  for  the  rendered  data  scatterplot  is  shown 
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Figure  39.  Bhattacharyya  coefficient  of  the  2  dimensional  histograms  representing 
the  correlation  between  measured  and  modeled  distributions  of  NDSI  vs  NDGRI  at 
illumination  angles  of  10,  30,  60,  and  85  degrees. 

in  Figure  38.  The  correlation  between  the  measured  and  modeled  distributions  of 
Figure  33  and  Figure  38  with  the  BC  for  each  of  our  illumination  angles  is  shown 
in  Figure  39.  The  correlation  is  best  at  10  and  85  degree  illumination,  but  there 
is  room  for  improvement.  Additional  rehnement  of  the  BRDF  modeling  parameters 
at  1080nm  and  1580nm  and  characterization  of  our  reference  panels  are  future  work 
described  in  the  next  section. 

4.3  Summary 

The  results  presented  in  Chapter  4  are  from  an  evolving  model  that  originated 
from  previous  research  for  the  subsurface  reflectance  in  [28],  with  added  work  from 
our  current  research  on  surface  reflectance.  Modeling  of  an  experimental  scenario 
was  accomplished  to  estimate  apparent  surface  reflectance  as  a  function  of  scene 
conhguration. 
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5.  Conclusion 


In  Chapter  5  our  summary  and  conclusions  are  given  of  the  work  accomplished 
on  skin  Bidirectional  Reflectance  Distribution  Function  (BRDF)  characterization 
and  reflectance  modeling.  Recommendations  for  future  work  related  to  BRDF  mea¬ 
surement  at  the  near  infrared  (NIR)  wavelengths  and  application  of  the  model  to 
improve  on  human  detection  are  discussed. 

Our  research  goals  were  to  characterize  the  BRDF  of  human  skin  for  the  wave¬ 
lengths  used  in  human  detection  and  incorporate  specular  reflection  into  the  existing 
diffuse  reflectance  model.  The  first  goal  was  met  with  a  combination  of  direct  measure¬ 
ment  and  extraction  from  a  test  scenario  with  the  Air  Force  Institute  of  Technology 
(AFIT)  Sensors  Exploitation  Research  Group  (SERG)  multispectral  system.  Since 
the  commercial  GASI®  system  was  setup  for  measurement  at  544nm  and  633nm,  the 
unpolarized  in-plane  BRDF  of  human  skin  was  characterized  for  incident  angles  of  0, 
30,  and  60  degrees.  A  method  was  developed  with  an  image-based  setup  to  charac¬ 
terize  BRDF  at  633nm  with  the  intent  to  move  into  measureing  lOSOnm  and  1580nm, 
but  a  reference  BRDF  of  the  labsphere  panels  at  those  wavelengths  is  needed.  Instead, 
the  NIR  wavelengths  were  fit  from  a  measurement  scenario  of  the  AFIT  SERG  multi¬ 
spectral  system.  The  Schlick  normal  reflectance  and  variance  parameters  were  found 
which  provide  a  reflectance  distribution  with  the  microfacet  model  for  any  incident 
angle  between  0  and  90  degrees.  The  second  goal  with  human  surface  modeling  in 
the  Blender®  environment.  Modeling  was  performed  for  illumination  angles  similar 
to  an  experimental  scenario  with  a  calculated  comparison.  We  now  discuss  future 
work  needed  in  this  area  and  conclusions  to  our  accomplishments. 
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5.1  Future  Work 


5.1.1  Dismount  Detection. 

While  the  goal  of  our  work  was  to  improve  on  the  skin  reflectance  model,  the 
eventual  application  is  to  better  detect  people.  In  Chapter  2,  the  Normalized  Differ¬ 
ence  Skin  Index  (NDSI)  and  Normalized  Difference  Green  Red  Index  (NDGRI)  were 
introduced  as  calculations  for  the  rules-based  detector.  Shown  in  our  research  was 
a  reflectance  model  of  skin  that  includes  a  specular  and  diffuse  components,  thereby 
realizing  a  more  accurate  representation  of  normalized  difference  (ND)  algorithms: 

_  [Pdj^l,  Oj)  +  Ps(Al,  9i,  9d)]  —  [Pd{^2,  Oj)  +  Ps{^2,  Oj,  Od)]  /p.  ,  .  N 

[Pdi^l,  ^i)  +  ^d)]  +  [Pd{^2,  ^i)  +  Ps{^2,  ^i,  ^d)] 

where  Ai  is  lOSOnm  and  A2  is  1580nm  for  NDSI,  or  Ai  is  633nm  and  A2  is  544nm 
for  NDGRI,  6i  is  the  global  incident  angle,  and  6d  is  the  global  viewing  angle.  If 
the  detection  system  is  viewing  into  the  specular  lobe,  a  large  difference  in  value 
for  the  two  wavelengths  is  less  likely  and  the  resulting  value  is  driven  closer  to  zero 
since  the  denominator  will  be  much  larger.  Future  work  may  be  interested  in  setting 
the  threshold  for  detection  with  modeling  estimations  based  on  source  and  detector 
angular  locations.  With  the  simulation  tool,  the  digitized  human  subject  is  setup 
in  any  orientation  to  a  source  and  camera  in  three  dimensional  space  to  estimate 
apparent  reflectance  for  the  rules-based  detector. 

5.1.2  Skin  Shade  Variation. 

While  observing  a  person  with  the  human  eye,  it  is  clear  that  some  people  are 
visibly  dark  shaded  and  others  are  light  shaded.  The  modeling  in  [28]  revealed  that 
the  subsurface  component  drives  the  effect  of  this  skin  type.  Meanwhile,  the  front 
surface  component  was  shown  to  add  or  subtract  to  this  effect  based  on  surface 
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geometry.  While  the  subsurface  component  is  matched  to  a  specific  person,  the  surface 
component  was  shown  to  change  as  the  geometry  of  the  individual  changes.  Since 
the  head  and  neck  are  most  commonly  exposed  and  are  similiarly  shaped  for  people, 
these  areas  were  the  focus  of  modeling  and  simulation  of  this  research.  Adjusting  the 
diffuse  value  in  the  model  for  the  respective  skin  type  allows  us  to  estimate  reflectance 
values  for  any  person.  Additional  measurements  and  modeling  with  darker  shader 
subjects  is  needed  to  validate  the  model  for  use  with  diferent  shaded  subjects. 

5.1.3  Sources  of  uncertainty. 

While  including  a  BRDF  model  showed  favorable  modeling  results,  additional 
work  is  needed  to  increase  accuracy.  The  BRDF  models  fit  at  lOSOnm  and  1580nm 
were  approximated  and  based  on  image  measurements,  but  an  accurate  BRDF  mea¬ 
surement  is  needed  to  refine  the  model  at  those  wavelengths.  Additionally,  due  to 
an  unknown  amount  of  specularity  from  our  reference  Spectralon  pannels,  some  error 
is  introdused  into  our  experimental  measurements.  Characterization  of  our  panels 
at  our  detection  wavelengths  is  needed  to  make  that  correction.  In  additional,  with 
better  panel  characterization  we  can  move  to  scenarios  that  incorporates  different 
camera  positions.  As  of  now,  moving  the  camera  position  into  a  specular  lobe  will 
add  additional  uncertainty  to  our  measurements. 

5.2  Conclusion 

As  was  shown  in  our  modeling  results,  the  rendered  images  that  included  a  specu¬ 
lar  modeling  component  correlated  more  favorably  with  the  similar  scenario  measured 
with  our  multispectral  system  than  just  the  diffuse  value  alone.  The  rendered  images 
run  through  the  rules-based  detector  proved  to  be  an  adequate  indicator  for  skin  pix¬ 
els.  We  increased  fidelity  in  our  method  of  skin  reflectance  estimation  from  a  single 
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diffuse  value  representing  each  of  the  wavelengths  used  in  detection  to  a  distribution 
of  values  across  the  human  subject. 
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Appendix  A.  Facet  Geometry 


To  determine  estimated  reflectance  through  the  combined  skin  reflectance  model, 
a  method  was  developed  to  work  with  representative  shape  surfaces  for  the  human 
body.  The  surfaces  are  facetized  by  assuming  a  small  angle  approximation  for  the 
projected  held  of  view  for  a  single  pixie  element  of  the  imageing  system.  In  this 
appendix  the  global  illumination  incident  angle  is  transformed  to  local  angles  for  the 
surfaces  demonstrated  in  this  research. 

For  a  single  flat  facet  in  Figure  15  whose  local  normal  aligns  with  the  global 
reference  normal,  the  following  symbols  representing  the  physical  configuration  are 
defined  as  follows: 

•  The  range  (R)  is  the  distance  from  the  source  area  to  the  detector. 

•  Area  of  the  optic  (A^)  for  calculating  the  solid  angle  of  the  detector. 

•  Facet  area  (A)  is  the  projected  detector  element  following  Equation  (2.2.3). 

•  Reflected  angle  {6d)  is  the  angular  difference  of  the  detector  from  the  reference 
axis. 

•  Incident  angle  {6i)  is  the  angular  difference  of  the  source  from  the  reference  axis. 

Three  cases  were  demonstrated  in  this  thesis  for  modeling  which  included  viewing 
outside  the  specular  lobe  which  resulted  in  diffuse  only  component  observed  and 
viewing  directly  into  the  specular  lobe  with  (0j)  equals  {dd)  in  the  corresponding 
reflection  quadrant. 

The  cylindrical  geometry  in  Figure  16  is  realized  with  the  equation  of  a  circle  with 
estimated  radius  of  the  appendage.  Most  of  the  symbols  are  similar  to  the  single  facet 
geometry,  but  the  global  angles  are  transformed  into  local  angles  for  each  facet  with 
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the  following  equations: 


local  ^i, global  ^rot 

a  -a  +  -ifR  +  r-x 

^r, local  oj-ot  tan  t  ) 

y 


where  the  symbols  are  dehned  as: 

•  Rotation  angle  {6rot)  is  the  angular  rotation  between  the  global  reference  axis 
and  the  local  facet  normal. 

•  Local  reflection  angle  {Or, local)  and  incident  angle  {O^ocai)  calculated  for  each 
projected  facet. 

•  Coordinates  on  the  cylinder  represented  by  x  and  y. 

•  The  radius  of  the  circle  is  r. 
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Appendix  B.  Spectralon  BRDF 


The  labsphere  spectralon  panels  have  been  used  for  all  calibration  in  this  research. 
In  [7] ,  spectralon  BRDF  was  characterized  with  the  CASI  system.  The  resulting  plots 
are  shown  in  Figure  40  for  633  and  3390nm  wavelengths  at  4  different  incident  angles. 
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Figure  40.  Measured  BRDF  of  Labsphere  spectralon  from  [7]  using  the  CASI®  system 
at  633nm  and  3390nm  for  incident  angles  of  (a)  0  degrees,  (b)  30  degrees,  (c)  60  degrees 
and  (d)  75  degrees. 
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Appendix  C.  Scene  Simulation 


This  appendix  demonstrates  a  method  for  useing  Blender  software  to  simulate 
a  remote  sensing  scenario  by  testing  a  simple  case  with  three  diffuse  panels.  The 
outer  two  panels  are  defined  as  the  reference  panels  with  reflectances  of  0.97  and  0.02 
respectively.  In  this  scenario  a  diffuse  panel  with  reflectance  of  0.4  is  placed  in  the 
center,  but  any  arbitrarily  shaped  surface  with  BRDF  characterization  can  take  its 
place.  For  this  test  the  source  and  camera  are  placed  normal  to  the  surface,  but 
these  objects  can  be  placed  anywhere  in  the  hemisphere  above  the  panels  for  future 
experiments. 

In  Figure  41  a  layout  of  this  configuration  is  displayed  where  the  outer  squares 
represent  the  camera  field  of  view  and  the  dot  above  the  center  panel  represent  the 
light  source.  The  blender  interface  allows  the  user  to  define  the  material  diffuse  and 
specular  characteristics.  For  the  specular  shader  there  are  several  predefined  BRDF 
models  in  Blender  with  adjustable  parameters  to  set  for  the  desired  response. 

With  a  rendered  image  of  the  scene  it  can  now  be  treated  as  any  picture  when 
applying  analysis  techniques  is  matlab.  So,  ELM  is  calculated  using  the  two  refer¬ 
ence  panels  to  convert  the  rendered  image  from  digital  space  into  reflectance  space. 
Figure  43  is  a  histogram  of  the  rendered  image  after  converted  into  reflectance  space. 
What  it  shows  is  that  the  values  assigned  in  blender  to  the  three  panels  are  mapped 
in  reflectance  space  with  an  error  of  ±  0.02. 


C-1 


Timelire  Game  Render  Help  I  »lSR:Z-Moael 


TiTl  I  » lSCE:Scene  ~  | X  E  vwav blenaer.org  249.2  Ve:18  |  Fa.4  |  0b:6-l  |  Lai  |  Mein.l.40M  {II  SZM)  |  Time;CO:00.61  |  Camera 


View  Select  Object  Mode  ^*|  I# -I  |fi 


la -I  -  ptoii  laaBKioa  l»l<l«IAI 


Figure  41.  Blender  interface  of  three  reference  panels  assigned  varying  diffuse  re¬ 
flectance  characteristics. 


Figure  42.  Rendered  image  of  three  reference  panels  wheresource  and  detector  are 
normal  to  the  surface. 
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Figure  43.  Histogram  of  reflectance  values  from  the  simulated  image  after  mapping 
from  digital  space  into  reflectance  space  with  ELM. 
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Appendix  D.  Image-Based  Measurement 


The  limitation  to  our  NIR  wavelengths  for  the  current  setup  of  the  CAST®  system 
requires  us  to  characterize  BRDF  at  NIR  wavelengths  with  an  alternative  method. 
In  Chapter  2,  some  novel  methods  for  an  image-based  BRDF  design  were  described. 
Those  concepts  are  the  influence  to  an  image-based  method  for  characterizing  in-plane 
BRDF  for  skin  at  our  NIR  wavelengths. 

An  image-based  method  with  the  Figure  14  conhguration  is  our  hrst  approach 
to  characterizing  BRDF  at  the  NIR  wavelengths.  The  method  conhgures  the  image- 
based  system  with  the  parameters  listed  in  Table  4,  to  dehne  the  range  from  the 
sample  to  the  detector,  source  and  detector  angles  with  respect  to  the  sample  normal, 
and  optic  area  for  the  collection.  The  extended  version  of  a  Goodrich  SIU  camera 
is  used  covering  400nm  to  1700nm.  For  the  source,  collimated  light  is  directed  to 
the  sample  location  from  an  off-axis  parabolic  mirror  (OAP).  The  OAF  is  an  optical 
component  that  is  a  section  of  a  full  parabolic  mirror  with  a  focal  point  of  304.8  mm. 
When  the  incoming  rays  from  the  laser  source  diverge  from  the  focal  point  of  the 
OAP,  they  collimate  upon  contact  with  its  surface  at  an  expanded  area.  A  snapshot 
of  the  system  layout  in  Figure  44  shows  these  components  with  a  Spectralon®  panel 
at  the  sample  location.  The  red  dashed  line  is  the  path  for  light  with  arrows  showing 
the  direction. 

The  image  data  from  the  Goodrich  camera  is  in  the  form  of  a  digital  number 
representing  the  intensity  of  radiation,  originating  from  a  laser  source,  reflecting  from 


Table  4.  Image-Based  Measurement  System  Parameters. 


Parameter 

Value 

Units 

Coptic 

19.60 

cm? 

R 

64.0 

cm 

Os 

0,  60 

degrees 

Od 

10,  20,  30,  40,  50,  60,  70,  80 

degrees 
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Figure  44.  Setup  of  the  image-based  measurement  system,  showing  the  illumination 
path  in  red  with  arrows  pointing  the  direction.  The  laser  source  is  aligned  so  that  the 
light  would  diverge  from  the  lens,  at  the  focal  point  of  the  OAP.  The  camera  is  attached 
to  an  arm  for  rotation  around  the  sample.  A  Spectralon  panel  rotated  45  degrees  from 
the  incoming  light  is  located  at  the  sample  location. 


the  hand  and  reaching  the  camera.  All  other  lighting  in  the  room  is  turned  off  during 
the  collection.  To  transform  the  digital  number  into  a  BRDF  value,  we  use  the  BRDF 
ELM  with  Equations  (3.3.1),  (3.3.2),  and  (3.3.3).  Two  Labsphere  Spectralon®  panels 
are  the  reference  standard  for  the  calculations.  The  BRDF  of  Spectralon®  at  633nm, 
shown  in  Appendix  B,  was  measured  at  AFIT  in  [7]  and  represents  the  reference 
BRDF  used  to  hnd  the  ELM  parameters. 

In  Figure  45,  the  image-based  measurement  of  the  Type  I/II  skin  is  compared 
along  with  the  CAST  measurement  for  0  and  60  degrees.  The  image-based  BRDF 
data  follows  the  same  trend  in  both  plots  but  may  differ  from  the  CASI®  data  with 
an  approximate  RMSE  of  9.4%  at  0  degrees  and  1.2%  at  60  degrees.  We  attribute 
most  error  to  the  source,  reference  material,  and  camera  angle  corrections.  First,  the 
collimated  light  is  variant  across  the  illuminated  hand  due  to  corrosion  on  the  OAP. 
The  reference  BRDF  from  [7]  is  used  to  represent  the  standard  BRDF  of  our  panels 
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(a)  (b) 

Figure  45.  The  image-based  BRDF  measurements  of  Type  I/II  skin  (blue)  compared 
with  the  CASI®  measurements  (red).  The  illumination  source  is  at  633nm  with  an 
incident  angle  of  (a)  0  deg  and  (b)  60  deg. 


which  may  be  different.  Finally,  while  the  camera  positions  are  set  mannally,  this 
introduces  a  ±3  degree  error  with  cosine  correction  for  the  increase  in  area  viewed  by 
an  instantaneous  pixel.  Furthermore,  since  we  are  unable  to  characterize  BRDF  of  our 
Spectralon®  panels  for  the  NIR  wavelengths,  the  direct  measurement  approach  is  not 
used  to  characterize  skin  at  our  NIR  wavelengths.  The  following  recommendations  are 
advised  for  improvement  with  the  system.  Characterize  our  reference  panel  BRDFs 
for  all  our  wavelengths  of  interest.  Improve  the  illuminating  source  to  bring  it  closer  to 
space  invariant  conditions.  Automate  the  rotating  arm  collection  system  to  increase 
angular  accuracy. 
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spectral  properties  of  skin  for  the  detection  and  classification  of  humans.  Since  then  a  multispectral  skin  detection  system 
was  developed  to  exploit  the  optical  properties  of  human  skin  at  wavelengths  in  the  visible  and  near  infrared  region  of 
the  electromagnetic  spectrum.  A  rules-based  detector,  analyzing  an  image  spectrally,  currently  bases  its  skin  pixel 
selection  criteria  on  a  diffuse  skin  reflectance  model.  However,  when  observing  skin  in  direct  view  of  the  sun,  a  glint  of 
light  off  skin  is  common  and  indicates  specularity.  The  areas  of  skin  with  a  high  degree  of  specular  reflectance,  results  in 
misdetections.  We  show  that  skin  is  characterized  by  diffuse  and  specular  reflectance,  with  both  components  dependent 
on  the  scene  configuration.  While  we  cannot  always  rely  on  the  person  to  directly  face  the  camera  or  have  constant 
illumination  conditions,  it  is  important  to  have  flexibility  with  the  rules-based  detector  as  the  scene  changes.  Our 
research  better  characterizes  skin  reflectance  as  a  function  of  source  and  detector  angular  locations  to  improve  on  the 
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